CN103531208A - Astronautic stress emotion identification method based on short-term memory weight fusion - Google Patents

Abstract

The invention discloses a method for identifying voice emotion under an astronautic stress environment based on short-term memory weight fusion. The method comprises the following steps: extracting the prior probability of the memory accuracy rate of a voice frame according to a short-term memory forgetting rule of psychology, wherein the prior probability of the memory accuracy rate and an adjusting coefficient jointly form importance weight of the frame; calculating the judgment fused weight of a voice segment based on the importance weight of each frame; performing emotion identification on each voice segment; performing emotion fusion according to the judgment fused weight to obtain the final identification result. Through the adoption of the method provided by the invention, the identification accuracy rate of a text-related speaker identification system can be effectively increased; in addition, the system has higher robustness on noise.

Description

A kind of space flight stress emotion identification method merging based on short-term memory weight

Technical field

The present invention relates to a kind of speech-emotion recognition method, particularly the speech-emotion recognition method merging based on short-term memory weight under a kind of space flight stress environment.

Background technology

In space flight particular surroundings, being emotionally stable of personnel has great importance with objective evaluation.The in the situation that of enclosure space, weightlessness and long-time uninteresting operation, easily bringing out irritated grade stress speech emotional.

Traditional speech emotional automatic identification technology mainly comprises two levels: the one, and which kind of feature in employing voice signal is as emotion recognition, the problem of affective feature extraction namely, the one, how by specific voice data classification, the problem of namely pattern-recognition.

In voice signal, the expression of emotion is always not fairly obvious.Different emotions can have similar feature, and different speakers can exist larger individual difference, and this research to affective characteristics has caused larger difficulty, or even people's ear is listened and distinguished and also caused very large difficulty.In the automatic identification of speech emotional, a very important problem is exactly to find the clearly expression of emotion in feature space, namely needs to carry out extraction and the optimization of affective characteristics.

Emotion modeling is a basic problem in speech emotional identification.We can adopt emotion class models or the dimensional space model in the theories of psychology to carry out modeling to emotion.Emotion class models is divided into mutually different classification by emotion, is a kind of discrete model.Dimensional space model is considered as the continuous variable in hyperspace by emotion, is a kind of continuous model.Gauss hybrid models is a kind of method of higher recognition performance that obtains in speech emotional identification in recent years.Because gauss hybrid models is stronger to the capability of fitting of data, learning ability that therefore may be in the larger data type such as languages identification, Speaker Identification, speech emotional identification " individual difference " is stronger.Yet the shortcoming of gauss hybrid models is to be also limited by training data, and successfully setting up emotion model need to have sufficient training data.

Summary of the invention

The present invention is directed to stress space environment in the demand of continuous speech emotion recognition, the recognition methods that the portion's emotion recognition of setting a trap is combined with overall emotion recognition.Propose a kind of speech-emotion recognition method that embeds short-term memory forgetting curve, its major technique step is:

According to the short-term memory forgetting law in psychology, extract the prior probability of the memory accuracy of speech frame; Recall the prior probability of accuracy, with the weights of importance of the common configuration frame of adjustment factor; Weights of importance based on each frame, weight is merged in the judgement of computing voice section; Each voice segments is carried out emotion recognition; According to judgement, merge weight, carry out emotional fusion, obtain final recognition result.

(1) from short-term memory forgetting law curve, extract the prior probability of recalling accuracy.

(1-1) short-term memory forgetting curve is sampled, the time in sampling interval is t, 0 < t < 18s;

(1-2) obtain k sample value d(k), as the prior probability of recalling accuracy.

(2) by recalling prior probability and the subjective adjustment factor of accuracy, obtain the weights of importance of time domain frame.

(2-1) subjective adjustment factor r is set, the reference value of r is 1, can (0,1] dynamic adjustments in scope;

(2-2) by taking advantage of sexual intercourse to obtain weights of importance f (n)=r * d (k) of time domain frame, wherein n is frame sequence.The frame memory accuracy more waning to the close is higher, and weight is higher, and it is zero that the frame beyond 18 seconds is recalled accuracy, and corresponding weights of importance is also zero.

(3) cut apart continuous voice signal, form emotion recognition unit.

(3-1) continuous voice signal is divided into the unit that is less than 18s, the persistence length reference value of unit is that 2s is to 9s.Dividing method be take naturally and to be paused and syllable is separatrix.

(3-2) extract frame by frame the feature of voice segments, comprise the parameters,acoustics such as fundamental tone, intensity, resonance peak.

(4), according to the weights of importance of every frame, obtain the fusion weight of emotion recognition unit.

(4-1) to cutting apart m the voice segments s (i) obtaining in (3) step, i=1,2 ... m, the weights of importance f of the frame that extraction frame sequence is separately corresponding respectively _i(n);

(4-2) calculate the fusion weight of each voice segments s (i)

(5) each emotion recognition unit is carried out to speech emotional identification.

(5-1) to training data, adopt gauss hybrid models to carry out emotion modeling to voice segments, obtain probability Distribution Model;

(5-2) input data are carried out to the emotion recognition based on bayesian criterion, obtain the emotion vector e (i) of every section of emotion recognition unit s (i)=[e(i, 1), e (i, 2) ..., e (i, p)], p is emotion categorical measure;

(5-3) according to merging weight w (i), fusion is adjudicated in each emotion recognition unit, final emotion be output as E=w (1) * e (1)+w (2) * e (2)+... + w (m) * e (m).

Advantage of the present invention and effect are:

1. the present invention, according to the short-term memory forgetting law in psychology, extracts the prior probability of the memory accuracy of speech frame; Recall the prior probability of accuracy, with the weights of importance of the common configuration frame of adjustment factor, weight is merged in the judgement of computing voice section, finally according to judgement, merge weight and identify, thereby realized the fusion of emotion, improved stress emotion recognition effect.

2. the present invention can detect effectively for the affective style relevant with cognitive process such as irritated, tired, self-confident, stress in space environment, have important application prospect, can to personnel's emotional stability, carry out early warning and monitoring timely.

Other advantages of the present invention and effect will continue to describe below.

Accompanying drawing explanation

Fig. 1---emotion recognition system flowchart

Fig. 2---latent structure method figure

The forgetting curve figure of Fig. 3---short-term memory

The weight iterativecurve of Fig. 4---gaussian component

Fig. 5---embed short-term memory and forget the gauss hybrid models recognition result before weight

Fig. 6---in the present invention, method stress speech emotional recognition result

Embodiment

Below in conjunction with drawings and Examples, technical solutions according to the invention are further elaborated.

Fig. 1 is speech emotional recognition system block diagram involved in the present invention, and wherein main module comprises: segmentation feature extracts, emotion model is trained, divide the calculating of frame weights of importance, local emotion recognition result, overall emotion Vector Fusion etc.The implementation method of lower mask body introducing system.

One. speech feature extraction

For identifying with the proper vector of modeling, generally there are two kinds of building methods, static statistics feature and in short-term behavioral characteristics.Behavioral characteristics is stronger to the dependence of phoneme information.The variation meeting of text has larger impact to affective characteristics.In the middle of emotional speech, roughly comprise three kinds of information sources, speaker information, semantic information and emotion information.At structure affective characteristics with when selecting feature, not only need to make the feature emotion information that reflects as much as possible, namely along with the variation of emotion, there is obvious variation, but also need to keep feature not to be subject to the semantic impact changing as far as possible.

As shown in Figure 2, the phonetic feature of employing has nature static and of overall importance to latent structure method.Feature has comprised prosodic features and tonequality feature, and wherein the computing formula of single order shake is as follows:

${\mathrm{Jitter}}^1=\frac{\frac{1}{N-1}\underset{i=1}{\overset{N-1}{\mathrm{\&Sigma;}}}|x_i-x_{i+1}|}{\frac{1}{N}\underset{i=1}{\overset{N}{\mathrm{\&Sigma;}}}{x}_i}\&times;100---\left(1\right)$

The computing formula of second order shake is as follows:

${\mathrm{Jitter}}^2=\frac{\frac{1}{N-2}\underset{i=2}{\overset{N-1}{\mathrm{\&Sigma;}}}|{2x}_i-x_{i+1}-x_{i-1}|}{\frac{1}{N}\underset{i=1}{\overset{N}{\mathrm{\&Sigma;}}}{x}_i}\&times;100---\left(2\right)$

When the frequency-division section feature of structure spectrum energy, do not adopt the energy percentage in 650Hz-4kHz, although this frequency range relates to the first resonance peak and most the second resonance peak, but the energy of this frequency range is subject to impact that content of text changes greatly, mainly along with the variation of phoneme information, changes.

In spectrum energy feature, also adopted energy percentage more than 4kHz, the increase of this band segment energy can reflect the raising of incentive degree, can be used for distinguishing sad and anger etc.

Harmonic noise is than the latent structure (feature 78 is to feature 95) that has increased too frequency-division section in feature.Because harmonic noise is than the impact that can be subject to noise, particularly in high band, the impact of noise is more obvious, therefore considers that frequency-division section structure harmonic noise is than feature, so that more careful description emotion changes the signal intensity bringing.In the division of frequency range, be divided into four following frequency ranges of frequency range: 400Hz (harmonic component that has comprised lower frequency), 400Hz-2000Hz frequency range (energy range that has roughly comprised the first two resonance peak), 2000Hz-5000Hz frequency range (harmonic component of upper frequency).5kHz is more serious with noise effect in the signal of super band, and also inapplicable for the lower language material of some sampling rates, does not therefore adopt.

Two. embed minute frame weight calculation of short-term memory forgetting curve

From short-term memory forgetting law curve h (τ), extract the prior probability p (τ) that recalls accuracy, short-term memory forgetting curve as shown in Figure 3.Short-term memory forgetting curve is sampled, and the time in sampling interval is t, 0 < t < 18s.Obtain k sample value d(k), as the prior probability of recalling accuracy.

p(τ)=d(τ/t) （3）

d(k)=h(kt) （4）

According to following mode, obtain the weights of importance of time domain frame:

f(n)=r×d(k),0<k<18s/t （5）

f(n)=0,k>18s/t （6）

Wherein r is subjective adjustment factor, 0 < r≤1, and wherein n is frame sequence.The frame memory accuracy more waning to the close is higher, and weight is higher, and the frame beyond 18 seconds is recalled accuracy and approached zero, and the weights of importance of correspondence is also zero.

To continuous speech signal s (τ), form emotion recognition cell S (m), m is paragraph numbering.

S(m)=s(τ _i)-s(τ _j),τ _j-τ _i<18s （7）

The persistence length reference value of unit is that 2s is to 9s.Dividing method be take naturally and to be paused and syllable is separatrix.

According to the weights of importance of every frame, obtain the fusion weight of emotion recognition unit

$\begin{array}{c}w\left(i\right)=\underset{i}{\mathrm{\&Sigma;}}{f}_i\left(n\right)\\ =\underset{i}{\mathrm{\&Sigma;r}}\&times;h\left(\mathrm{kt}\right)\end{array}---\left(8\right)$

Three. gauss hybrid models modeling

Gauss hybrid models (Gaussian Mixture Model, GMM) can define by following formula:

$p\left(X_t\right|\mathrm{\&lambda;})=\underset{i=1}{\overset{M}{\mathrm{\&Sigma;}}}{a}_i{b}_i\left(X_t\right)---\left(9\right)$

Here X is the D dimensional feature vector of speech samples, and t is its sample sequence number; b _i(X), i=1,2 ..., M is member's density; a _i, i=1,2 ..., M is mixed weight-value.Each member's density be a D dimension variable about mean value vector U _iwith covariance matrix Σ _igaussian function, form is as follows:

$b_i\left(X_t\right)=\frac{1}{{\left(2\mathrm{\&pi;}\right)}^{D/2}{\left|{\mathrm{\&Sigma;}}_i\right|}^{1/2}}\exp \{-\frac{1}{2}{(X_t-U_i)}^{\&prime;}{\mathrm{\&Sigma;}}_i^{-1}(X_t-U_i)\}---\left(10\right)$

Wherein mixed weight-value satisfies condition:

$\underset{i=1}{\overset{M}{\mathrm{\&Sigma;}}}{a}_i=1---\left(11\right)$

The mixed density of complete Gauss is by mean value vector, covariance matrix and the mixed weight-value parametrization of all member's density.These parameters gatherings are expressed as together:

λ _i={ a _i, U _i, Σ _i, i=1,2 ..., M (12) is according to bayes decision criterion, and the emotion recognition based on GMM can obtain by maximum a posteriori probability,

$\mathrm{EmotionLabel}=\underset{k}{\arg \max }\left(p\left(X_t\right|{\mathrm{\&lambda;}}_k)\right)---\left(13\right)$

Wherein k is emotion classification sequence number.

For the parameter estimation of gauss hybrid models, can adopt EM(Expectation-maximization) algorithm carries out.EM is greatest hope algorithm, and its basic thought is since an initialized model λ, removes to estimate a new model

make

this stylish model becomes initial model for repetitive operation next time, and this process is carried out repeatedly until reach convergence threshold.This is similar to for estimating the Baum-Welch revaluation algorithm of Hidden Markov Model (HMM) (HMM) parameter.During the EM of each step repeats, following revaluation formula guarantees the likelihood value monotone increasing of model:

The revaluation of hybrid parameter:

$a_m^i=\frac{\underset{t=1}{\overset{T}{\mathrm{\&Sigma;}}}{\mathrm{\&gamma;}}_{\mathrm{tm}}^i}{\underset{t=1}{\overset{T}{\mathrm{\&Sigma;}}}\underset{m=1}{\overset{M}{\mathrm{\&Sigma;}}}{\mathrm{\&gamma;}}_{\mathrm{tm}}^i}---\left(14\right)$

The revaluation of mean value vector:

${\mathrm{\&mu;}}_m^i=\frac{\underset{t=1}{\overset{T}{\mathrm{\&Sigma;}}}{\mathrm{\&gamma;}}_{\mathrm{tm}}^i{X}_t}{\underset{t=1}{\overset{T}{\mathrm{\&Sigma;}}}{\mathrm{\&gamma;}}_{\mathrm{tm}}^i}---\left(15\right)$

The revaluation of variance matrix:

${\mathrm{\&Sigma;}}_m^i=\frac{\underset{t=1}{\overset{T}{\mathrm{\&Sigma;}}}{\mathrm{\&gamma;}}_{\mathrm{tm}}^i(X_t-{\mathrm{\&mu;}}_m^i){(X_t-{\mathrm{\&mu;}}_m^i)}^{\&prime;}}{\underset{t=1}{\overset{T}{\mathrm{\&Sigma;}}}{\mathrm{\&gamma;}}_{\mathrm{tm}}^i}---\left(16\right)$

${\mathrm{\&gamma;}}_{\mathrm{tm}}^i=\frac{a_m^{i-1}N\left(X_t\right|{\mathrm{\&mu;}}_m^{i-1},{\mathrm{\&Sigma;}}_m^{i-1})}{\underset{m=1}{\overset{M}{\mathrm{\&Sigma;}}}{a}_m^{i-1}N\left(X_t\right|{\mathrm{\&mu;}}_m^{i-1},{\mathrm{\&Sigma;}}_m^{i-1})}---\left(17\right)$

The estimated value of the weight of each component of GMM, average and covariance matrix, is tending towards convergence by iteration each time.

Degree of mixing in gauss hybrid models, can only derive in theory a fixing scope, concrete value need to be determined in experiment, the weight of each gaussian component can be estimated to obtain by EM algorithm, in the iteration of EM algorithm, avoid covariance matrix to become singular matrix, guarantee convergence.

Take weight as example, and the iterativecurve of EM algorithm as shown in Figure 4, has demonstrated the convergence situation of iteration each time in figure.Ordinate represents the numerical value of the weight of each gaussian component, and horizontal ordinate represents the iteration optimization number of times of EM algorithm, and the curve of different colours and shape represents different gaussian component.Wherein part Gaussian mixture components is zero, illustrates that degree of mixing arranges higher.Initial value is to be obtained by the initialization of K mean cluster, after 35 left and right of iteration, and algorithm convergence.

Four. identification by stages and fusion

Local emotion recognition result is carried out to overall emotional fusion, is no more than the capacity 18 seconds of short-term memory:

The output emotion vector of every section of emotion recognition unit s (i)

e(i)=[e（i,1）,e(i,2),…,e(i,p)] （18）

P is emotion categorical measure, according to merging weight w (i), fusion is adjudicated in each emotion recognition unit, and final emotion is output as

E=w(1)×e(1)+w(2)×e(2)+…+w(m)×e(m) （19）

E is final emotion recognition result, and m is local voice segments sequence number.

Five. in emotion recognition, embed continuously the compliance test result of short-term memory forgetting curve weight

Select continuous emotion language material to test.The sample that has comprised five kinds of affective states such as irritated, happy, tired, confidence and neutral state in experimental data, has retained the serial number information of recording collection site between statement sample, adjacent statement can obtain by adjacent serial number.Every kind of emotion language material comprises 1000 paragraph samples, amounts to 5000.The discrimination of continuous speech emotion recognition as shown in Figure 5, embeds after short-term memory forgetting curve, by weight fusion method, improved stress emotion recognition effect, as shown in the recognition result in Fig. 6.For the affective style relevant with cognitive process such as irritated, tired, self-confident, detect, stress in space environment, have important application prospect, can to personnel's emotional stability, carry out early warning and monitoring timely.

The scope that the present invention asks for protection is not limited only to the description of this embodiment.

Claims (6)

1. the space flight stress emotion identification method merging based on short-term memory weight, is characterized in that comprising the following steps:

Step 1 extracts the prior probability of recalling accuracy from short-term memory forgetting law curve;

Step 2, by recalling prior probability and the subjective adjustment factor of accuracy, obtains the weights of importance of time domain frame;

Step 3, cuts apart continuous voice signal, forms emotion recognition unit;

Step 4, according to the weights of importance of every frame, obtains the fusion weight of emotion recognition unit;

Step 5, carries out speech emotional identification to each emotion recognition unit.

2. the space flight stress emotion identification method merging based on short-term memory weight according to claim 1, is characterized in that, described step 1 specifically comprises the following steps:

Step 1-1, samples to short-term memory forgetting curve, and the time in sampling interval is t, 0 < t < 18s;

Step 1-2, obtains k sample value d(k), as the prior probability of recalling accuracy;

3. the space flight stress emotion identification method merging based on short-term memory weight according to claim 1, is characterized in that, described step 2 specifically comprises the following steps:

Step 2-1, arranges subjective adjustment factor r, and the reference value of r is 1, can (0,1] dynamic adjustments in scope;

Step 2-2, by taking advantage of sexual intercourse to obtain weights of importance f (n)=r * d (k) of time domain frame, wherein n is frame sequence; The frame memory accuracy more waning to the close is higher, and weight is higher, and it is zero that the frame beyond 18 seconds is recalled accuracy, and corresponding weights of importance is also zero;

4. the space flight stress emotion identification method merging based on short-term memory weight according to claim 1, is characterized in that, described step 3 specifically comprises the following steps:

Step 3-1, is divided into continuous voice signal the unit that is less than 18s, and the persistence length reference value of unit is that 2s is to 9s; Dividing method be take naturally and to be paused and syllable is separatrix;

Step 3-2, extracts the feature of voice segments frame by frame, comprises the parameters,acoustics such as fundamental tone, intensity, resonance peak.

5. the space flight stress emotion identification method merging based on short-term memory weight according to claim 1, is characterized in that, described step 4 specifically comprises the following steps:

Step 4-1, to cutting apart m the voice segments s (i) obtaining in step 3 claimed in claim 1, i=1,2 ... m, the weights of importance f of the frame that extraction frame sequence is separately corresponding respectively _i(n);

Step 4-2, calculates the fusion weight of each voice segments s (i)

6. the space flight stress emotion identification method merging based on short-term memory weight according to claim 1, is characterized in that, described step 5 specifically comprises the following steps:

Step 5-1, adopts gauss hybrid models to carry out emotion modeling to voice segments to training data, obtains probability Distribution Model;

Step 5-2, carries out the emotion recognition based on bayesian criterion to input data, obtain the emotion vector e (i) of every section of emotion recognition unit s (i)=[e(i, 1), e (i, 2) ..., e (i, p)], p is emotion categorical measure;

Step 5-3, according to merging weight w (i), adjudicates fusion to each emotion recognition unit, final emotion be output as E=w (1) * e (1)+w (2) * e (2)+... + w (m) * e (m).

Images