KR20180078252A - Method of forming excitation signal of parametric speech synthesis system based on gesture pulse model - Google Patents

Abstract

A system and method for forming an excitation signal of a parametric speech synthesis system based on a sentence pulse model is provided. The excitation signal may be formed using a plurality of subband templates instead of a single subband template. A plurality of subband templates may be combined to form an excitation signal, wherein the rate at which the template is added is dynamically based on the determined energy coefficient. These coefficients can vary from frame to frame and are also learned along with spectral parameters during feature training. Each coefficient is added to a feature vector containing spectral parameters and modeled using an HMM, and the excitation signal is determined.

Description

Method of forming excitation signal of parametric speech synthesis system based on gesture pulse model

The present invention generally relates to speech synthesis as well as communication systems and methods. More particularly, the present invention relates to the formation of excitation signals in a statistical parametric speech synthesis system based on a hidden Markov model.

A system and method for forming an excitation signal of a parametric speech synthesis system based on a sentence pulse model is provided. The excitation signal may be formed using a plurality of subband templates instead of a single subband template. A plurality of subband templates may be combined to form an excitation signal, wherein the rate at which the template is added is dynamically based on the determined energy coefficient. These coefficients can vary from frame to frame and are also learned along with spectral parameters during feature training. Each coefficient is added to a feature vector containing spectral parameters and modeled using an HMM, and the excitation signal is determined.

In one embodiment, a method for generating a parametric model for use in training a speech synthesis system is provided, wherein the system includes at least one training text corpus, a speech database, and a model training module The method comprising the steps of: obtaining, by the model training module, speech data for the training text corpus, the speech data comprising a recorded speech signal and a corresponding transcript; Translating the training text corpus into a context dependent phoneme label by the model training module; Extracting, by the model training module, at least one of a spectral characteristic, a plurality of band excitation energy coefficients, and a fundamental frequency value for each speech frame in the speech signal from the speech training database; Using the model training module to form a feature vector stream for each voice frame using at least one of the spectral characteristics, the plurality of band excitation energy coefficients, and the fundamental frequency value; Labeling the speech using a context dependent phoneme; Extracting a duration of each context dependent phoneme from the labeled speech; Performing a parameter estimation of the speech signal, the estimation of the parameter being performed including a feature, an HMM, and a decision tree; And identifying a plurality of subband eigenvalue pulse comprising a separate model for use in forming an excitation during synthesis.

In another embodiment, a method is provided for identifying a subband eigenvalue pulse from a verbal pulse database for speech synthesis system training, the method comprising: receiving a pulse from the vernal pulse database; Decomposing each pulse into a plurality of subband components; Dividing the subband component into a plurality of databases based on the decomposing step; Determining a vector representation of each database; Determining, from the vector representation, an eigenvalue pulse value for each database; And selecting the best eigenvalue pulse for each database for use in synthesis.

1 is a diagram illustrating an embodiment of a hidden Markov model-based text-to-speech system.
2 is a flow chart illustrating an embodiment of a feature vector extraction process.
3 is a flow chart illustrating an embodiment of a feature vector extraction process.
4 is a flow chart illustrating one embodiment of an eigenvalue pulse identification process.
5 is a flow chart illustrating an embodiment of a speech synthesis process.
[Cross reference to related application]
The present application was filed on May 28, 2014, entitled " Method for Forming a Excitation Signal for a Parametric Speech Synthesis System Based on a Glottal Pulse Model " ), Which is hereby incorporated by reference in its entirety.

In order to facilitate understanding of the principles of the present invention, reference will be made to the embodiments shown in the drawings and specific reference will be made to the drawings. It will nevertheless be understood that such description is not intended to limit the scope of the invention. It will be understood by those skilled in the art that various changes and modifications may be made to the embodiments described herein without departing from the spirit and scope of the invention as defined in the appended claims. Lt; RTI ID = 0.0 > a < / RTI >

와 같은 일정 지속 기간만큼 이전 수열에서 분리되며, 여기에서

는 피치 주기를 나타내고,

는 기본 주파수를 나타낸다. 비발화 영역(unvoiced region)에서, 각 수열은 백색 잡음(white noise)으로 모델링된다. 그러나 발화 영역(voiced region)에서, 여기(excitation)는 실제로는 임펄스 수열이 아니다. 대신, 여기는 성대 주름(vocal folds)의 진동 및 그 형상에 의해서 생성되는 음원 펄스의 서열이다. 또한, 펄스의 형상은 화자, 화자의 기분, 언어학적 맥락, 감정 등과 같은 다양한 요인에 따라서 변화될 수 있다.In speech synthesis, excitation is generally considered to be a quasi-periodic impulse sequence for the speech region. Each sequence,

, ≪ / RTI > where < RTI ID = 0.0 >

Represents a pitch period,

Represents the fundamental frequency. In the unvoiced region, each sequence is modeled as white noise. However, in the voiced region, the excitation is not actually an impulse sequence. Instead, this is the sequence of the source pulses generated by the vibrations of the vocal folds and their shape. Further, the shape of the pulse can be changed according to various factors such as the speaker, the speaker's mood, the linguistic context, emotion, and the like.

The source pulses are mathematically represented by length normalization and impulse alignment (via resampling), as described, for example, in European Patent EP 2242045 (Registered June 27, 2012, inventor Thomas Drugman et al. It is treated as a vector. The final length of the normalized source pulse signal is resampled to meet the target pitch. The source pulses are not selected in the database and are obtained through a series of calculations that impair the pulse characteristics of the frequency domain. Traditionally, sound source pulses were modeled using acoustic parameters or excitation models for an HMM-based system. However, this model interpolates / resamples the residual / And thus the characteristic of the model pulse in the frequency domain is impaired. Although other methods have been used in the standard way of selecting pulses, other methods have transformed residual pulses into vectors of equal length by length normalization. These methods also perform PCA on these vectors, which allows the final pulse to be selected as the calculated pulse, rather than the final pulse directly selected from the training data.

In contrast to computation, to obtain the final pulse directly from the training data, you can define a metric and also model the gating pulse by defining a vector representation. Excitation and pulses of the fundamental frequency are provided, excitation is also provided which does not resample or interpolate the pulse.

In statistical parametric speech synthesis, the speech unit signal is represented by a set of parameters that can be used to synthesize speech. This parameter can be learned, for example, by a statistical model such as an HMM. In one embodiment, the speech may be represented by a source-filter model, where the source / excitation is a signal that produces a given sound as it passes through the appropriate filter. 1 is a diagram illustrating an embodiment of a text-to-speech system based on a Hidden Markov Model (HMM) indicated generally by the reference numeral 100. FIG. One embodiment of an exemplary system may include, for example, two steps of training and combining, and these steps will now be described in more detail.

The voice database 105 may include a large amount of voice data used in voice synthesis. The voice data may include a recorded voice signal and a corresponding transcription. During the training phase, the speech signal 106 is converted to a parameter. The parameters may include excitation parameters, F0 parameters, and spectral parameters. The excitation parameter extraction 110a, the spectrum parameter extraction 110b and the F0 parameter extraction 110c are generated from the speech signal 106 delivered from the speech database 105. The hidden Markov model can be trained using the training module 115 using these extracted parameters and the label 107 from the voice database 105. [ Any number of HMM models can be generated from this training and these context dependent HMMs are stored in the database 120.

The synthesis step begins by creating a parameter 135 using context-dependent HMM 120. The parameter generation 135 may use the input value of the text (125) corpus to analyze the speech. Prior to use in parameter generation 135, analysis 130 of text 125 may be performed. During analysis 130, a label 131 is extracted from the text 125 and used in the parameter generation 135. In one embodiment, excitation parameters and spectral parameters may be generated in the parameter generation module 135.

The excitation parameter may be used to generate an excitation signal 140 that is input to the synthesis filter 145 along with the spectral parameter. The filter parameters are generally Mel frequency cepstral coefficients (MFCC) and are often modeled by a statistical time series using HMMs. The excitation signal is generated from the fundamental frequency value and the MFCC value to form a filter, thereby synthesizing the filter using the fundamental frequency as the time series value and the predicted value of the filter. The synthesized voice 150 is generated when the excitation signal passes through the filter.

The formation of the excitation signal 140 in FIG. 1 is essential to the quality of the output, or synthesized speech 150. In general, the spectral parameters used in statistical parametric speech synthesis systems include MCEPS, MGC, Mel-LPC, or Mel-LSP. In one embodiment, the spectral parameter is a mel-generalized cepstral (MGC) calculated from the pre-emphasized speech signal, but the zero-order energy coefficient is calculated from the original speech signal. In a conventional system, the fundamental frequency value alone is regarded as a source parameter and the entire spectrum is regarded as a system parameter. However, the spectral tilt or gross spectral shape in the speech spectrum is actually a characteristic of the gating pulse and is therefore considered as the source parameter. The spectral slope is collected and modeled for the glottal pulse-based excitation, but excluded as a system parameter. Instead, the spectral parameter (MGC) excluding the zero-order energy coefficient (speech energy) is calculated using the pre-emphasized speech. This coefficient varies slowly over time and can also be regarded as a prosodic parameter calculated directly from the raw speech.

Training and model building

FIG. 2 is a flow chart illustrating one embodiment of a feature vector extraction process, indicated generally at 200. This process can be performed during the spectral parameter extraction 110b of Fig. As mentioned above, parameters can be used in model training, such as HMM model training.

In step 205, a voice signal is received and converted to a parameter. As shown in Fig. 1, a voice signal can be received from the voice database 105. Fig. The control moves to steps 210 and 220 and the process 200 continues. In one embodiment, steps 210 and 215 are performed concurrently with step 220, and each result is also passed to step 225.

In step 210, a pre-emphasis on the speech signal is performed. For example, if the speech signal is pre-emphasized at this stage, low frequency source information is not collected in the determination of the MGC coefficient in the next step. Then control transfers to step 215 and process 200 continues.

In step 215, the spectral parameters for each voice frame are determined. In one embodiment, MGC coefficients 1 through 39 may be determined for each frame. Alternatively, MFCC and LSP may also be used. The control moves to step 225 and the process 200 continues.

In step 220, a zero-order coefficient for each voice frame is determined. In one embodiment, the determination of this coefficient may be determined using the raw speech in contrast to the pre-emphasized speech. The control moves to step 225 and the process 200 continues.

At step 225, the coefficients from steps 220 and 215 are added to the MGC coefficients of 1 to 39 to form 39 coefficients for each voice frame. The spectral coefficients for one frame may be referred to below as a spectral vector. The process 200 is terminated.

FIG. 3 is a flow chart illustrating an embodiment of a feature vector extraction process generally indicated at 300. This process can be performed during the excitation parameter extraction step 110a of FIG. As mentioned above, parameters can be used in model training, such as HMM model training.

In step 305, a voice signal is received and converted to a parameter. As shown in Fig. 1, a voice signal can be received from the voice database 105. Fig. Control transfers to steps 310, 320, and 325 and process 300 continues.

In step 310, a pre-emphasis is performed on the speech signal. For example, if the speech signal is pre-emphasized at this stage, low frequency source information is not collected in the determination of the MGC coefficient in the next step. Control moves to step 315 and process 300 continues.

In step 315, linear predictive coding or LPC analysis is performed on the pre-emphasized speech signal. For example, LPC analysis generates the coefficients used in the next step of performing the inverse filtering. Control moves to step 320 and process 300 continues.

In step 320, an inverse filtering is performed on the analyzed signal and the original speech signal. In one embodiment, if pre-emphasis (step 310) is not performed, step 320 is not performed. Control moves to step 330 and process 300 continues.

In step 325, a fundamental frequency value is determined from the original speech signal. The fundamental frequency value may be determined using any standard technique known in the art. Control moves to step 330 and process 300 continues.

In step 330, the glottal cycle is segmented. Control moves to step 335 and process 300 continues.

In step 335, the gatekeeper cycle is decomposed. For each frame, in one embodiment, the corresponding grammar cycle is decomposed into subband components. In one embodiment, the subband component may comprise a plurality of bands, where each band may comprise a bottom and top component.

In the spectrum of a typical screen pulse, there is a high energy bulge at low frequencies and a typical flat structure at high frequencies. The demarcation between these bands varies per pulse as well as the energy ratio. Given a sentence pulse, the cutoff frequency separating the high and low bands is determined. In one embodiment, the ZFR method can be used with the window size appropriately adjusted, but this technique can be applied to the spectral magnitude. A zero crossing is obtained at the edge of the low frequency bulge, which is considered to be the fractional frequency between the lower band and the higher band. The two components of the time domain can be obtained by placing a zero in the highband region of the spectrum before taking the inverse FFT to obtain a time domain version of the low frequency component of the loudspeaker pulse and vice versa to obtain the high frequency component . Control moves to step 340 and process 300 continues.

In step 340, the energy for the subband component is determined. For example, the energy of each subband component may be determined to form an energy factor for each frame. In one embodiment, the number of subband components may be two. Determination of the energy for the subband components may be done using any standard technique known in the art. The energy coefficients of the frame are collectively referred to below as an energy vector. Process 300 ends.

In one embodiment, two band energy coefficients for each frame are determined from the inverse filtered speech. The energy coefficient can represent the dynamic properties of glottal excitation. The inverse filtered speech has an approximation to the source signal and is then divided into a grammar cycle. The two band energy coefficients contain the energy coefficients of the low and high band components corresponding to the gate cycle in the source signal. The energy of the low frequency component includes the energy coefficient of the low band and the energy of the high frequency component similarly includes the energy coefficient of the high band. These coefficients may be modeled by including them in the feature vector of the corresponding frame, and then modeled using the HMM-GMM of the HTS.

In a non-limiting example, the two band energy coefficients of the source signal are added to the spectral parameters determined in process 200 to form a feature stream with the fundamental frequency value, which is a typical HMM-GMM (HTS ) Based TTS system. This model may be used in the speech synthesis process 500, as described below.

Eigenvalue pulse identification training

4 is a flow chart illustrating one embodiment of an eigenvalue pulse identification process, For each subband signature pulse database, an eigenvalue pulse may be identified, which may be used for synthesis as described in more detail below.

으로 나타낼 수 있다. 성문 펄스는 병합(pool)되어 데이터베이스를 생성한다. 단계(410)로 제어가 이동하며 프로세스(400)는 계속된다.At step 405, a sentence pulse database is created. In one embodiment, the verbal pulse database is automatically generated using training data (voice data) obtained from a voice actor. Given a speech signal s (n), a linear prediction analysis is performed. And performs an inverse filtering on the signal s (n) to obtain an integrated linear prediction residual signal, which is an approximation of the signal excitation. The integrated linear prediction residuals are then divided into gating cycles using techniques such as, for example, zero frequency filtering. A number of small signals, commonly referred to as gate signal pulses, are obtained,

. The gate pulse is pooled to create the database. The control moves to step 410 and the process 400 continues.

In step 410, pulses from the database are decomposed into subband components. In one embodiment, the sentence pulse may be decomposed into a plurality of subband speech pulses, such as low and high band components, and two band energy coefficients. In a typical gate pulse spectrum, there is a high energy bulge at low frequencies and a typical flat structure at high frequencies. However, the fraction between the bands varies per pulse as well as the energy ratio between these two bands fluctuates. As a result, different models for all of these bands may be required.

Given a gate pulse, the cutoff frequency is determined. In one embodiment, the cut-off frequency is to separate the high and low bands using a Zero Frequency Resonator (ZRF) technique while properly adjusting the window size, but this technique can be applied to spectral magnitudes . A zero crossing is obtained at the edge of the low frequency bulge, which is considered to be the fractional frequency between the low and high bands. The two components of the time domain can be obtained by placing a zero in the highband region of the spectrum before taking the inverse FFT to obtain a time domain version of the low frequency component of the loudspeaker pulse and vice versa to obtain the high frequency component . Control moves to step 415 and process 400 continues.

In step 415, a pulse database is formed. For example, in step 410, a plurality of loudspeaker pulse databases such as a low-band speech pulse database and a high-band speech pulse database are generated. In one embodiment, a number of databases corresponding to the number of bands formed are formed. Control moves to step 420 and process 400 continues.

At step 420, the vector representation of each database is determined. In one embodiment, two separate models for the low and high band components of the sentence pulse are generated, but the same method applies for each of these models and will be described below. In this context, the subband speech pulse refers to one component of the speech pulse.

The subband signal pulse signal space can be treated as a new mathematical metric space and will be described below.

)을 가정한다.

와

가 동일하고,

가 시간 변환/지연 버전이라면 상술한 공간 내에서의 변환이 식별된다.

와

가 주어지고, 여기에서

와

는 임의의 두 개의 서브 밴드 성문 펄스를 나타내고,

인 실수 상수가 존재하여

가 되고, 여기에서

가

의 힐베르트 변환인 경우,

는

와 동치인 상기 공간에 대해서, 동치 관계가 적용된다.The function space of a function with continuous and steady-state variation and unit energy (

).

Wow

Lt; / RTI >

Is a time-converted / delayed version, the transformation within the space described above is identified.

Wow

Is given, and here

Wow

Lt; / RTI > represents any two subband speech pulse,

There is a real constant

, And here

end

In the case of the Hilbert transform,

The

The same relation is applied to the space equivalent to the above.

)에 대해서 거리 척도(

)가 정의될 수 있다.

이라면, 두 함수 간의 정규화된 상호 상관 관계는

로 나타낼 수 있다.

가

의 힐베르트 변환일 때

로 둔다.

및

간의 각도는

로 정의되며, 여기에서

는 함수(

)의 최댓값으로 간주된다.

,

간의 거리 척도는

가 된다. 함수 공간(

)과 함께 척도(

)는 척도 공간(

)을 형성한다.Function space

) Was measured on a distance scale (

) Can be defined.

, Then the normalized cross-correlation between the two functions is

.

end

In the Hilbert Transformation of

.

And

The angle between

Lt; RTI ID = 0.0 >

Is a function (

) Is considered to be the maximum value.

,

The distance scale between

. Function space

) And the scale (

) Is the measure space (

).

)가 힐베르트 척도(Hilbertian metric)인 경우, 상술한 공간은 힐베르트 공간 내에 등방적으로 매장(embed)될 수 있다. 따라서, 함수 공간 내의 주어진 신호에 대해서,

는 힐베르트 공간 내의 벡터(

)에 사상(mapping)될 수 있으며, 이는 다음과 같이 나타낼 수 있다.Measure(

) Is a Hilbertian metric, the above space may be isotropically buried in the Hilbert space. Thus, for a given signal in the function space,

Is a vector within Hilbert space (

), Which can be expressed as follows.

은

에서의 고정 원소이다. 제로(0) 원소는

와 같이 표현된다. 사상(

)은 힐베르트 공간 내의 전체합을 나타낸다. 이 사상은 등거리 사상으로,

임을 의미한다.From here,

silver

. The zero (0) element

. thought(

) Represents the total sum in the Hilbert space. This idea is equidistant,

.

)에 대한 벡터 표현(

)은 해당 척도 공간 내의 모든 다른 신호로부터의

의 거리 집합에 따라서 결정된다. 척도 공간 내의 모든 다른 지점으로부터의 거리를 결정하는 것은 비실용적이며, 따라서 벡터 표현은 척도 공간의 일련의 고정된 개수의 지점(

)으로부터의 거리만에 의해서 결정될 수 있고, 이들 거리는 척도 공간으로부터의 다량의 신호에 대해서 척도 기반 클러스터링을 수행한 이후에 중심(centroid)으로서 획득되는 거리이다. 단계(425)로 제어가 이동하며 프로세스(400)는 계속된다.The signal given in the scale space (

) ≪ / RTI >

) From all other signals in the measure space

Is determined according to the set of distances. It is impractical to determine the distance from all other points in the scale space so that the vector representation is a series of fixed numbers of points in the scale space

), And these distances are the distances obtained as a centroid after performing the scale-based clustering for a large number of signals from the scale space. The control moves to step 425 and the process 400 continues.

및

) 간의 척도 또는 거리 개념(

)을 정의한다. 두 펄스(

,

) 간의 척도는 다음과 같이 정의된다.

,

간의 정규화된 원형 교차 상관 관계는 다음 수학식과 같이 정의된다.In step 425, an eigen pulse is determined and process 400 is terminated. In one embodiment, in order to determine the metric of the subband speech pulse, any two subband speech signal pulses

And

) Scale or distance concept (

). Two pulses (

,

) Is defined as follows.

,

The normalized circular cross-correlation between the points is defined by the following equation.

,

의 길이 중에서 가장 긴 것으로 정해진다. 더 짧은 신호는 척도를 계산하기 위해서 제로 확장(zero extend)되며 데이터베이스에서는 수정되지 않는다.

의 이산 힐베르트 변환(

)이 결정된다.The cycle of the cyclic correlation is

,

Is the longest among the lengths of the first and second lines. The shorter signal is zero extended to calculate the scale and is not modified in the database.

Discrete Hilbert transform of

) Is determined.

Next, a signal is obtained by the following equation.

,

) 간의 각도(

)의 코사인(cosine)은 다음과 같이 정의될 수 있다.Two signals (

,

)

) Can be defined as follows.

는 모든 샘플 신호(

) 중의 최댓값을 나타낸다. 거리 척도는 다음과 같을 수 있다.From here,

Lt; RTI ID = 0.0 > (

). The distance scale can be:

평균 클러스터링 알고리즘을 변형하여 전체 성문 펄스 데이터베이스(

)로부터의

클러스터 중심 성문 펄스(

cluster centroid glottal pulse)가 결정될 수 있다. 제 1 변형은 유클리드 거리 척도(Euclidean distance metric)를 상술한 바와 같이 성문 펄스에 대해서 정의된 척도(

)로 치환하는 단계를 포함한다. 제 2 변형은 클러스터의 중심을 업데이트하는 단계를 포함한다. 각 원소가

으로 표시되는 성문 펄스 클러스터의 중심 성문 펄스가 해당 원소(

)가 되는In the present invention,

The average clustering algorithm was modified to use the entire sentence pulse database (

)

Cluster centered field pulse (

cluster centroid glottal pulse) can be determined. The first variant measures the Euclidean distance metric on a scale defined for the sentence pulse (as described above)

). ≪ / RTI > The second variant includes updating the center of the cluster. Each element

The center gate pulse of the glottal pulse cluster represented by the corresponding element (

)

에서 최소가 된다. 클러스터링의 반복은

번째 클러스터의 임의의 중심 중에 더 이상의 시프트(shift)가 없으면 종료된다.silver

. The iteration of clustering

Th cluster, if there is no further shift in any center of the first cluster.

)가 주어지고, 또한 상술한 바와 같이 클러스터링에 의해서 결정된 중심 성문 펄스가

이라고 가정하면, 성문 펄스 데이터베이스의 크기는

이 된다. 각각의 성문 펄스를 거리 척도(distance metric)에 기초하여 클러스터(

)의 중심 중의 하나에 할당하면, 중심(

)에 할당된 원소의 전체 개수는

로 정의될 수 있다. 여기에서,

는 데이터베이스로부터 선택된 고정된 서브 밴드 성문 펄스를 나타내며, 그 벡터 표현은 다음과 같이 정의될 수 있다.The vector representation for the subband speech pulse may then be determined. Seongam Pulse

), And as described above, a central sentence pulse determined by clustering

, The size of the gate pulse database is

. Each sentence pulse is divided into clusters based on a distance metric

), The center (

) Is the total number of elements assigned to

. ≪ / RTI > From here,

Denotes a fixed subband plaintext pulse selected from the database, the vector representation of which can be defined as follows.

가 서브 밴드 성문 펄스(

)에 대한 벡터 표현인 경우,

는 다음과 같을 수 있다.From here,

Lt; RTI ID = 0.0 >

), ≪ / RTI >

Can be as follows.

For every sentence pulse in the database, the corresponding vector is determined and stored in the database.

PCA is performed on the vector space, and the eigenvalue pulse is identified. Principal component analysis (PCA) is performed on the vector set associated with the sentence pulse database to obtain an eigenvector. Subtract the mean vector of the entire vector database from each vector to obtain a mean subtracted vector. The eigenvalue vector of the covariance matrix of the vector set is then determined. As each eigenvector vector is obtained, the average difference vector is associated with a sentence pulse having a minimum Euclidean distance from the eigenvalue vector, and is also referred to as the corresponding eigenvalue pulse. Thus, an eigenvalue pulse for each subband signature pulse database is determined and one of each is selected based on a listening test, and the eigenvalue pulse can also be used in the synthesis as described in more detail below.

Use in speech synthesis

FIG. 5 is a flow chart illustrating an embodiment of a process for speech synthesis, indicated generally by the reference numeral 500. FIG. This process can be used to train the model obtained in process 100 (see FIG. 1). In one embodiment, a sentence pulse used as an excitation in a particular pitch cycle scales each sentence pulse for a corresponding two band energy coefficients, and then a low-pass word template pulse and a high-band word template pulse . The two band energy coefficients for a particular cycle are determined by the band energy coefficients of the frame corresponding to the corresponding pitch cycle. This is formed from the sentence pulses and is also filtered to obtain the output speech.

The synthesis may be performed in the frequency domain and the time domain. In the frequency domain, for each pitch period, the corresponding spectral parameter vector is transformed into a spectrum and also multiplied by the spectral pulse spectrum. Applying a Discrete Fourier Transform (DFT) to the result, a speech segment corresponding to the pitch cycle is obtained. Apply overlap to all acquired pitch synchronization timings in the time domain to obtain synchronized speech.

In the time domain, an excitation signal is established and filtered using a Mel Log Spectrum Approximation (MLSA) filter to obtain a synthesized speech signal. The given gating pulse is normalized to the unit energy. For the non-firing region, the white noise of fixed energy is located in the excitation signal. For the firing region, the excitation signal is initialized to zero (0). For example, the pitch boundary is calculated using the fundamental frequency value provided every 5 ms frame. An envelope pulse is placed in order from all pitch boundaries and an overlap is added to the excitation signal that is zero initialized to acquire the signal. Overlap addition is performed on the speech pulse at each pitch boundary and adds a band pass filtered low amount of quantitative white noise to ensure that a small amount of random / probable components are present in the excitation signal. To suppress wind noise in synthesized speech, a suturing mechanism is applied where multiple excitation signals are formed using a right shifted pitch boundary and a circularly shifted gentle pulse. The right-shift at the pitch boundary used in the model construction includes a fixed constant, and the sentence pulses used in the model construction are shifted left by the same amount in a circle. Finally, the stitched excitation is the arithmetic mean of the excitation signal. When the average value is passed through the MLSA filter, a voice signal is obtained.

At step 505, text is entered into the model of the speech synthesis system. For example, the model (context-dependent HMM 120) obtained in FIG. 1 provides features that are used to receive the input text and then synthesize the voice associated with the input text, as described below. The control moves to step 510 and step 515 and the process 500 continues.

In step 510, a feature vector for each frame is predicted. This prediction can be performed using standard methods in the art to which the present invention belongs, such as, for example, a context-dependent decision tree. Then control transfers to steps 525 and 540 and step 500 continues.

In step 515, the fundamental frequency value (s) is determined. Control transfers to step 520 and process 500 continues.

In step 520, a pitch boundary is determined. The control moves to step 560 and the process 500 continues.

In step 525, the MGC is determined for each frame. For example, 0 to 39 MGCs are determined. Control transfers to step 530 and process 500 continues.

In step 530, the MGC is transformed into a spectrum. Control is transferred to step 535 and process 500 continues.

In step 540, an energy coefficient for each frame is determined. The control moves to step 545 and the process 500 continues.

At step 545, an Eigen pulse is determined and normalized. The control moves to step 550 and the process 500 continues.

In step 550, an FFT is applied. Control is transferred to step 535 and process 500 continues.

At step 535, a multiplication of the data may be performed. For example, the data from step 550 is multiplied with the data in step 535. [ In one embodiment, this step may be performed by sample-by-sample multiplication. The control moves to step 555 and the process 500 continues.

In step 555, an inverse FFT is applied. The control moves to step 560 and the process 500 continues.

At step 560, an overlap add is performed on the speech signal. Control transfers to step 565 and process 500 continues.

At step 565, the speech signal output is received and the process 500 ends.

Although the present invention has been shown and described in detail in the drawings and foregoing description of the invention, such illustration and description are to be considered illustrative and exemplary, and not restrictive, It is to be understood that it is desired that all equivalents, variations, and alterations included in the spirit of the invention as described in the claims and / or in the following claims are to be protected.

Accordingly, the appended claims should be accorded the broadest interpretation so as to encompass all such modifications as well as all such equivalents to the one set forth in the drawings, and the equivalents of those described herein, and the appropriate scope of the invention should be determined.

Claims (19)

A method for generating a parametric model used in training of a speech synthesis system, the system comprising at least one training text corpus, a speech database, and a model training module,
(A) by the model training module, obtaining voice data for the training text corpus, the voice data comprising a recorded voice signal and a corresponding voice record;
(B) converting, by the model training module, the training text corpus into a context dependent phoneme label;
(C) extracting, by the model training module, at least one of a spectral characteristic, a plurality of band excitation energy coefficients, and a fundamental frequency value for each speech frame in the speech signal from the speech training database;
(D) forming, by the model training module, a feature vector stream for each voice frame using at least one of the spectral characteristics, a plurality of band excitation energy coefficients, and a fundamental frequency value;
(E) labeling the speech using a context-dependent phoneme;
(F) extracting the duration of each context dependent phoneme from the labeled speech;
(G) parameter estimation of the speech signal, the estimation of the parameters being performed including a feature, an HMM, and a decision tree; And
(A) identifying a plurality of subband eigenvalue sign pulses including a separate model for use in forming an excitation during synthesis,
Way.
The method according to claim 1,
The spectral characteristics include,
(A) determining an energy coefficient from the speech signal;
(B) pre-emphasizing the speech signal and determining an MGC coefficient for each frame of the pre-emphasized speech signal;
(C) adding the energy coefficient and the MGC coefficient to form an MCG for each frame of the signal; And
(D) extracting a spectral vector for each frame,
Way.
The method according to claim 1,
Wherein the plurality of band excitation energy coefficients are selected from the group consisting of:
(A) determining, from the speech signal, a fundamental frequency value;
(B) performing pre-emphasis on the speech signal;
(C) performing LCP analysis on the pre-emphasized speech signal;
(D) performing inverse filtering on the speech signal and the LCP analyzed signal;
(E) partitioning the sentence cycle using the base frequency value and the inverse filtered speech signal;
(F) decomposing a corresponding grammar cycle into subband components for each frame;
(G) calculating energy of each subband component to form a plurality of energy coefficients for each frame; And
(A) using the energy coefficients to extract an excitation vector for each frame,
Way.
The method of claim 3,
Wherein the subband component comprises at least two bands,
Way.
The method of claim 4,
Wherein the subband component comprises at least a highband component and a lowband component,
Way.
The method according to claim 1,
Wherein identifying a plurality of subband eigenvalue word pulses comprises:
(A) generating a voiceprint pulse database using the voice data;
(B) decomposing each pulse into a plurality of subband components;
(C) dividing the subband component into a plurality of databases based on the decomposing step;
(D) determining a vector representation of each database;
(E) determining an eigenvalue pulse value for each database from the vector representation; And
(F) selecting the best eigenvalue pulse for each database for use in synthesis; and
Way.
The method of claim 6,
Wherein the plurality of subband components comprise a low band and a high band,
Way.
The method of claim 6,
The gender database includes:
(A) performing a linear prediction analysis on a speech signal;
(B) performing inverse filtering of the signal to obtain an integrated linear prediction residual; And
(C) dividing the integrated linear prediction residual into a lull word cycle to obtain a plurality of lull word pulses,
Way.
The method of claim 6,
Wherein said disassembling comprises:
(A) a cut-off frequency, the cut-off frequency separating and grouping the subband components;
(B) obtaining a zero crossing at the edge of the low frequency bulge;
(C) placing a zero in the highband region of the spectrum and obtaining a time domain version of the low frequency component of the sentence pulse, the obtaining comprising performing an inverse FFT; And
(D) placing a zero in the low-band region of the spectrum prior to obtaining the time domain version of the high-frequency component of the vernal pulse, wherein the obtaining comprises performing an inverse FFT Further comprising:
Way.
The method of claim 9,
The grouping includes a low band grouping and a high band grouping.
Way.
The method of claim 9,
The step of separating and grouping the subband components is performed using the ZFR method with a window of the appropriate size and applied to the spectrum size,
Way.
The method of claim 6,
Wherein the step of determining a vector representation of each database comprises:
Further comprising a series of distances from a series of fixed number of points in the scale space obtained as a centroid after performing scale-based clustering for a large number of signals from the scale space.
Way.
A method for identifying a subband eigenvalue pulse from a speech frame pulse database for speech synthesis system training,
(A) receiving a pulse from the vernal pulse database;
(B) decomposing each pulse into a plurality of subband components;
(C) dividing the subband component into a plurality of databases based on the decomposing step;
(D) determining a vector representation of each database;
(E) determining, from the vector representation, an eigenvalue pulse value for each database; And
(F) selecting the best eigenvalue pulse for each database for use in synthesis.
Way.
14. The method of claim 13,
Wherein the plurality of subband components comprise a low band and a high band,
Way.
14. The method of claim 13,
Wherein the vernier pulse database comprises:
(A) performing a linear prediction analysis on a speech signal;
(B) performing inverse filtering on the signal to obtain an integrated linear prediction residual; And
(C) dividing the integrated linear prediction residual into a lull word cycle to obtain a plurality of lull word pulses,
Way.
14. The method of claim 13,
Wherein said disassembling comprises:
(A) a cut-off frequency, the cut-off frequency separating and grouping the subband components;
(B) obtaining a zero crossing at the edge of the low frequency bulge;
(C) placing a zero in the high-band region of the spectrum prior to obtaining the time domain version of the low-frequency component of the speech pulse, the obtaining comprising performing an inverse FFT Includes; And
(D) placing a zero in the low-band region of the spectrum prior to obtaining the time domain version of the high-frequency component of the vernal pulse, wherein the obtaining comprises performing an inverse FFT Further comprising:
Way.
18. The method of claim 16,
The grouping includes a low band grouping and a high band grouping.
Way.
18. The method of claim 16,
The step of separating and grouping the subband components is performed using the ZFR method with a window of the appropriate size and applied to the spectrum size,
Way.
14. The method of claim 13,
Wherein the step of determining a vector representation of each database comprises:
Further comprising a series of distances from a series of fixed number of points in the scale space obtained as a centroid after performing scale-based clustering for a large number of signals from the scale space.
Way.

Images