KR102323482B1 - Conversation agent system and method using emotional history - Google Patents

Abstract

The present technology relates to a dialogue agent system and method using speech emotion history. According to a specific example of the present technology, learning data is derived by reflecting the accumulated discount value for inducing emotional change for a large number of collected original data, and learning is performed based on the learning technique set for the derived learning data, and the conversation model and generates a response sentence and response emotion for each input speech sentence and speech emotion using the rescued dialogue model, converts the generated response emotion and response sentence into a natural language form, and combines them to utter the It is possible to improve the accuracy of emotions and to conduct effective conversations reflecting emotions.

Description

CONVERSATION AGENT SYSTEM AND METHOD USING EMOTIONAL HISTORY

The present invention relates to a dialog agent system and method using a speech emotion history, and more particularly, to a conversation in which the speaker's emotion is actively reflected as a response sentence is generated by reflecting the accumulated discount value for inducing a change in the speaker's emotion. It's about the technology that made it possible.

In the existing interactive system research, the user's emotions were not taken into account and they were rushed to answer the uttered sentences, but recently, research to develop an interactive system including emotions is being actively conducted.

The deep learning encoder applied to such an interactive system can show excellent performance in the field of emotion classification by using deep learning technology to express variable-length documents as fixed-length document vectors.

However, in the case of an LSTM (Long Short Term Memory) encoding device, which considers the final output of the entire document sequence as a document vector, the recognition rate of the initially input pattern rapidly decreases as the input length increases, making it suitable as an encoding device for long documents. There is a problem that has not been done.

The present invention can improve the accuracy of conversation by generating and uttering response sentences and response emotions reflecting the discount accumulated compensation value that induces emotional changes of the speaker, and using the speech emotion history that can perform an effective conversation in which emotions are reflected An object of the present invention is to provide a dialogue agent system and method.

The object of the present invention is not limited to the above-mentioned object, and other objects and advantages of the present invention not mentioned may be understood by the following description, and will be more clearly understood by the embodiments of the present invention. It will also be readily apparent that the objects and advantages of the present invention can be realized by means of the means and combinations thereof indicated in the claims.

Using the speech emotion history according to an embodiment, the dialogue agent system

a dialogue model building apparatus for calculating a cumulative discount reward value for inducing a change in the speaker's emotions with respect to a plurality of collected original data, and constructing a dialogue model by reflecting the calculated cumulative discount compensation value;

a receiver for receiving sentences and emotions included in utterances;

a preprocessor for preprocessing the extracted sentences and emotions of the receiving device into one learning data;

a response generating device for deriving response sentences and emotions, respectively, based on a dialogue model built for one learning material of the preprocessor; and

It may be characterized in that it comprises an output device that converts each of the derived response sentences and emotions into a natural language form and then combines them to combine the response emotions for the utterance emotions and the response sentences for the utterance sentences to utter them.

Preferably, the dialogue model building device comprises:

an original data collection module for collecting a plurality of original data;

a discount cumulative compensation value calculation module for calculating a discount cumulative compensation value for each of the original data;

a learning module that generates learning materials by reflecting the accumulated discount value calculated on the respective original materials, and then performs learning on the generated learning materials based on a predetermined learning algorithm; and

It may include a dialog model building module for building a dialog model based on the learning result.

Preferably, the discount cumulative compensation value calculation module comprises:

The discount calculation reward value is initially set to reward=0,

If index+2n+2<2, it is determined whether the emotion of the episode of x[index +2n+2] is happiness, and if it is happy, the current discount calculation compensation value = the previous discount calculation compensation value+r ⁿ ,

After increasing to n=n+1, it may be provided to repeatedly perform all original data.

Here, r is the discount rate, x[index+2n+2] is an episode of one original data with sentences and emotions, index is the identification information of the original data, and n is the learning data and the original corrected with the discount cumulative reward value. distance information from the data.

Preferably, the discount cumulative compensation value increases based on the previous discount cumulative compensation value when a sentence of the same emotion is uttered immediately following a response of a random response sentence of the original data, and a predetermined number of responses to a random response sentence of the original data When the sentence of the same emotion is uttered after the passage of

Preferably, the dialogue agent system using the speech emotion history

A model update device for updating the dialog model by learning the input speech sentences and emotions and the output response sentences and emotions with policy gradient training based on a predetermined reinforcement learning policy (Policy) may further include.

The conversation agent method using the speech emotion history of an embodiment includes:

a dialogue model construction step of calculating a cumulative discount reward value for inducing a change in the speaker's emotions with respect to a plurality of collected original data, and constructing a dialogue model by reflecting the calculated cumulative discount compensation value;

a receiving step of receiving sentences and emotions included in the utterance;

a pre-processing step of pre-processing the extracted sentences and emotions into one learning data;

a response generating step of deriving a response sentence and an emotion based on a dialogue model built for the one learning material, respectively; and

Conversation agent method using speech emotion history, which includes an output step of converting each of the derived response sentences and emotions into a natural language form and then combining them to combine the response emotions for the speech emotions and the response sentences for the speech sentences. A conversation agent method using the speech emotion history is characterized in that it is one.

Preferably, the dialogue model building step comprises:

Collecting a large number of original data,

Calculate the discount cumulative compensation value for each of the original data,

After generating learning materials by reflecting the accumulated discount value calculated on the respective original materials, learning is performed on the generated learning materials based on a predetermined learning algorithm,

It may be provided to build a dialogue model based on the learning result.

Preferably, the discount cumulative compensation value is

Initial set reward=0,

If index+2n+2<2, it is determined whether the emotion of the episode of x[index +2n+2] is happiness, and if it is happy, the current discount calculation compensation value = the previous discount calculation compensation value+r ⁿ ,

After increasing to n=n+1, it may be provided to repeatedly perform all original data.

Here, r is the discount rate, x[index+2n+2] is an episode of one original data with sentences and emotions, index is the identification information of the original data, and n is the learning data and the original corrected with the discount cumulative reward value. distance information from the data.

The discount cumulative compensation value increases based on the previous discount cumulative compensation value when a sentence of the same emotion is uttered immediately following the response of a random response sentence of the original data, and a predetermined number of past When the sentence of the same emotion is uttered afterward, it may be provided to decrease based on the previous discount accumulated compensation value.

Preferably after the output step

A model update step of updating the dialog model by learning the input utterance sentences and emotions and the output response sentences and emotions with policy gradient training based on a predetermined reinforcement learning policy (Policy) may further include.

According to an embodiment, the learning data is derived by reflecting the accumulated discount value for inducing emotional change for a plurality of collected original data, and the conversation model is rescued by performing learning based on the learning technique set for the derived learning data. Using the rescued dialogue model, a response sentence and a response emotion for each inputted speech sentence and speech emotion are generated, the generated response emotion and response text are converted into a natural language form, and then combined and uttered to the speaker's emotion. It is possible to improve the accuracy of communication and to conduct effective conversations reflecting emotions.

The following drawings attached to the present specification illustrate preferred embodiments of the present invention, and serve to further understand the technical spirit of the present invention together with the detailed description of the present invention to be described later, so the present invention is a matter described in such drawings should not be construed as being limited only to
1 is a system configuration diagram according to an embodiment.
2 is a detailed configuration diagram of an apparatus for constructing a dialogue model of a system according to an embodiment.
3 is an exemplary view showing the original material and the learning material according to an embodiment.
4 is an overall flowchart illustrating an operation process of a system according to an embodiment.
5 is a detailed flowchart of a dialog model building process according to an embodiment.

The present invention applies to an interactive system. However, the present invention is not limited thereto, and may be applied to all interactive communication systems and methods to which the technical spirit of the present invention can be applied.

It should be noted that technical terms used herein are used only to describe specific embodiments, and are not intended to limit the present invention. In addition, the technical terms used in this specification should be interpreted in the meaning generally understood by those of ordinary skill in the art to which the present invention belongs, unless otherwise defined in this specification, and excessively inclusive. It should not be construed in the meaning of a human being or in an excessively reduced meaning. In addition, when the technical terms used in the present specification are incorrect technical terms that do not accurately express the spirit of the present invention, they should be understood by being replaced with technical terms that can be correctly understood by those skilled in the art. In addition, general terms used in the present invention should be interpreted as defined in advance or according to the context before and after, and should not be interpreted in an excessively reduced meaning.

Also, as used herein, the singular expression includes the plural expression unless the context clearly dictates otherwise. In the present application, terms such as "consisting of" or "comprising" should not be construed as necessarily including all of the various components or various steps described in the specification, some of which components or some steps are It should be construed that it may not include, or may further include additional components or steps.

Also, terms including ordinal numbers such as first, second, etc. used herein may be used to describe various elements, but the elements should not be limited by the terms. The above terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, a first component may be referred to as a second component, and similarly, the second component may also be referred to as a first component.

When a component is referred to as being “connected” or “connected” to another component, it may be directly connected or connected to the other component, but another component may exist in between. On the other hand, when it is mentioned that a certain element is "directly connected" or "directly connected" to another element, it should be understood that no other element is present in the middle.

Hereinafter, a preferred embodiment according to the present invention will be described in detail with reference to the accompanying drawings, but the same or similar components are assigned the same reference numerals regardless of reference numerals, and redundant description thereof will be omitted. In addition, in the description of the present invention, if it is determined that a detailed description of a related known technology may obscure the gist of the present invention, the detailed description thereof will be omitted. In addition, it should be noted that the accompanying drawings are only for easy understanding of the spirit of the present invention, and should not be construed as limiting the spirit of the present invention by the accompanying drawings. The spirit of the present invention should be construed as extending to all changes, equivalents, or substitutes other than the accompanying drawings.

An embodiment generates and generates a response word and a response emotion for each word and emotion included in the speaker's sentence by using a conversation model built with learning data that reflects the discount accumulated compensation value derived based on the original data between the speaker and the speaker. Each of the response emotions and response words is converted into natural language, and then the combined response sentence is uttered.

1 is a diagram showing the configuration of a dialogue agent system using an emotion history according to an embodiment, FIG. 2 is a diagram showing a detailed configuration of the dialogue model building apparatus 100 shown in FIG. 1, FIG. 3 is FIG. It is a diagram showing the original data and learning data collected to build the illustrated dialogue model.

1 to 3 , the system according to an embodiment includes a dialog model building device 100 , a receiving device 200 , a preprocessing device 300 , a response generating device 400 , and an output device 500 . may include.

The dialog model building apparatus 100 derives learning materials by reflecting the accumulated discount value for inducing a change in the speaker's emotions for a plurality of collected original data, and builds a dialog model through learning on the derived learning data. As shown in FIG. 2 , the dialog model building device 100 includes an original data collection module 111 , a discount cumulative compensation value calculation module 112 , a learning module 113 , and a dialog model building module ( 114) may be included.

The original data collection module 111 may collect a plurality of original data, and the collected original data may be transmitted to the discount accumulation compensation value calculation module 112 . Here, as shown in FIG. 3A , one original data is represented by sentences and emotions included in the spoken sentences between the interlocutors.

The discount cumulative compensation value calculation module 112 may calculate a discount cumulative compensation value (reward) based on a predetermined loss function from at least one sentence and emotion following the utterance of the collected original data.

The discount cumulative reward value (reward) can be calculated by deriving the solution of the loss function with the following algorithm.

(1) Initial setting of discount calculation reward value reward=0

(2) If index+2n+2< 2, it is determined whether the emotion of the episode of x[index +2n+2] is happiness, and if it is happy, current discount calculation compensation value = previous discount calculation compensation value+r ⁿ

(3) After increasing to n=n+1, proceed to step (2)

Here, r is the discount rate and has a value less than 1, and x[index+2n+2] is one original data with sentences and emotions. Here, if x is an odd number, it is the original data uttered by the speaker 1 as an odd-numbered sentence, and if x is an even number, it is the original data uttered by the speaker 2 as an odd-numbered sentence.

In addition, the index is an index of the original data, and it is possible to derive a discount cumulative reward value for any two consecutive episodes. This indicates the number of utterance sentences in episode x during the conversation. For example, since the values of index and n of the first utterance sentence are 0, the first episode x[index+2n+2] for deriving the discount cumulative reward value is x[2]. If the episode of x[2] includes the emotion of happiness, the accumulated discount reward value is the previous discount operation reward value + r ⁿ . Here, n has a value of 0.

That is, n is a variable for calculating the cumulative discount reward value, and represents the distance from the utterance sentence to the emotion for calculating the cumulative discount compensation value. The distance n = 0 to the first sentence rewarded with The distance n=2 to the second sentence rewarded with the cumulative reward value discounted to the emotion.

In addition, if the emotion contained in the immediately following sentence is happiness as a response to a random sentence of the original data, the discount cumulative reward value may be increased based on the previous discount cumulative compensation value, and a predetermined number of times as a response to the random sentence, the next original data If the emotion included in the sentence is happiness, the accumulated discount reward value may be increased based on the previous discount accumulated reward value.

The learning material of FIG. 3B to which the calculated accumulated discount reward value is reflected is transmitted to the learning module 313 , and the learning module 113 may perform learning on the learning material to which the accumulated discount accumulated reward value is reflected. Here, in an embodiment, learning may be performed in various forms, such as machine learning or deep learning, but is not limited thereto.

Then, the learning result of the learning module 113 is transmitted to the dialog model building module 114 , and the dialog model building module 114 may build a dialog model based on the learning result. This conversation model is a recurrent neural network encoder and decoder model, and a series of processes for building a recurrent neural network encoder and decoder model has been filed by the present applicant. Accordingly, the dialogue model building module 114 derives a discounted cumulative reward for each input sentence and an output sentence, multiplies the discounted cumulative compensation by a weighting factor of a predetermined loss function to derive a final loss function, and the final loss The final dialog model can be built by learning the original data input by the gradient descent algorithm using the function, and the response word and response emotion for each speech sentence and emotion input based on the final dialog model built on this can be derived.

On the other hand, the receiver 200 performs a function of receiving the speaker's sentence, for example, when the speaker says "I'm so happy", converts the speaker's voice into a sentence and transmits it to the preprocessor 300 .

Here, in one embodiment, for convenience of description, the receiver 200 is described as an example of a voice converter that simply converts the speaker's voice into a sentence, but extracts emotions in the various forms described above and delivers them to the preprocessor 300 . can

In addition, the receiving device 200 may extract the speaker's emotions through the speaker's facial expression or behavior recognition, etc., and recognize the speaker's voice strength and low in which the emotions included in the received word are reflected to recognize the speaker's emotions. can be extracted.

That is, by using the speaker's facial expression recognition algorithm, fast signals such as facial shapes that change according to the movement of facial muscles, changes in eyes, nose, and mouth, and temporary wrinkles are extracted, and the speaker's emotions are derived from the extracted fast signals. . Here, emotion means surprise, fear, disgust, anger, happiness, and sadness. That is, surprise has the shortest duration, fear is felt before harm is done, and disgust manifests itself as an antipathy towards something. Anger is the most dangerous emotion and is caused by frustration, threats, or stimulation. On the other hand, happiness is the most positive emotion, and sadness is characterized by a long duration due to loss. The emotion extracted with these characteristics is transmitted to the preprocessor 300 .

On the other hand, the receiver 200 uses at least one of Hidden Markov Models (HMM), Classification and Regression Trees (CART), and Stacked Sequential Learning (SSL) methods to estimate the prosody boundary of the speaker's voice to extract the speaker's emotions or Based on the frequency domain and magnitude analysis results for each emotion, the speaker's emotion is extracted and transmitted to the preprocessor 300 .

Hereinafter, in the present embodiment, for convenience of explanation, the speech-to-text converter is used to convert the speaker's voice into a word form, and then, the converted word and emotion included in the word are transmitted to the preprocessor 300 as an example. do.

Accordingly, the preprocessor 300 performs a function of dividing the received speaker's sentence into morpheme units and outputting the word in the morpheme form and the emotion included in the separated word. _{For example, the preprocessor 200 outputs the words (x 1} to x ₄ ,) of "I", "too much", and "I'm happy" and the emotion e_x of "happiness" which is the user's emotion.

And, the emotion is derived using the emotion TOBI (Tones and Breaking Indices: operation transcription rule), etc., and the derived emotion (e_x) is added to the tail of the word representing the emotion and transmitted to the response generating device 400 .

For example, the response generating device 400 performs a function of generating a response sentence and a response emotion based on the above-described final dialogue model with respect to the uttered sentence and emotion processed by the pre-processing unit 300, so that the response generating device ( 400) outputs a response emotion for each speech emotion and a response sentence for the speech sentence. Each of these response emotions and response sentences is transmitted to the output device 500 .

The output device 500 converts the received response emotion and response sentence into a natural language form, and combines them to generate a response sentence and utters the generated response sentence. For example, various response sentences and response emotions such as "I'm happy" are outputted, which are uttered by the interlocutor.

In addition, the output device 500 may output a response sentence and emotion in which emotions are reflected in various forms. For example, the output unit 500 may output and/or utter a response sentence to a character, such as an avatar, with a facial expression and/or action matched with a response emotion, and a voice strength and low pitch adjusted.

Accordingly, in one embodiment, learning materials are derived by reflecting the accumulated discount value for inducing emotional changes with respect to a plurality of collected original data, and learning is performed based on the set learning technique for the derived learning materials to build a conversation model, , using the constructed dialogue model to generate response sentences and response emotions for each input speech sentence and speech emotion, convert the generated response emotion and response sentence into a natural language form, and combine and utter the emotion of the speaker. Accuracy can be improved, and effective conversations reflecting emotions can be conducted.

Meanwhile, the dialog agent system using the emotion history according to an embodiment may further include a model updater 600 . The model updater 600, which has received the sentences and emotions of the preprocessing unit 300, is a policy gradient training method based on a reinforcement learning policy (Policy), and the input speech sentences and emotions and the output response sentences and The conversation model of the conversation model building apparatus 100 may be updated by learning about the emotion.

Accordingly, the dialog model building device 100 learns the sentences and emotions of the preprocessor 200 based on the policy change learning method and updates the learning results in the built dialog model. As the emotions become happier, the accumulated discount value can increase

In an embodiment, the emotion is described as happiness as an example, but it is applicable to a plurality of emotions, and is not limited thereto.

FIG. 4 is a flowchart illustrating an operation of a dialog agent system using an emotion history to have a conversation in response to an utterance according to an embodiment, and FIG. 5 is a detailed flowchart of the dialog model building apparatus 100 of FIG. 4 . A computer-readable recording medium in which a program for executing the dialog agent method using the emotion history according to an embodiment is recorded may be provided. The program may include at least one of an application program storing an item recommendation method, a device driver, firmware, middleware, a dynamic link library (DLL), and an applet. The conversation agent system using the emotion history may include a processor, and the processor may execute the conversation agent method using the emotion history by reading a recording medium in which the conversation agent method using the emotion history is recorded.

Referring to FIGS. 4 and 5 , in step 10, the apparatus 100 for constructing a dialogue model of the dialogue agent system using the speech emotion history provides a discount accumulated compensation value for inducing a change in the speaker's emotion with respect to the collected original data. can be derived.

For example, after collecting a plurality of original data in step 10 , the model building apparatus 100 may calculate a discount cumulative compensation value for the collected original data.

That is, the model building apparatus 100 collects a number of original data in step 11, and then sets the initial discount cumulative reward reward value to 0 for each original data in step 12, then step 13 It is determined whether index+2n+2<2 in step 13, and if index+2n+2<2 is satisfied as a result of determination in step 13, in step 14, the emotion of the episode of x[index +2n+2] is happy. judge whether

In addition, when the emotion of the episode of x[index +2n+2] is happy in step 14 , the dialog model building apparatus 100 performs the current discount calculation compensation value = previous discount calculation compensation value+r ^{n in step 15 .} n = n+1 increases to n=n+1 in step 16 and then proceeds to step 13 to determine whether index+2n+2<2.

Accordingly, learning data is derived by reflecting the discount accumulated reward value for the original data input in step 17, and learning is performed using a preset machine learning technique for the learning data derived in step 18, and learning is performed in step 19 ), a conversation model can be built based on the learning results.

Meanwhile, in step 20 , the receiver 200 of the dialog agent system using the speech emotion history may receive sentences and emotions included in the speech. In this case, the sentence may be derived based on the voice converter, and the emotion may be extracted through the emotion analyzer.

And in step 30, the preprocessor 300 may preprocess the extracted sentences and emotions as one learning material.

Meanwhile, in step 40 , the response generating apparatus 400 may derive a response sentence and emotion based on a conversation model built for one learning material, respectively.

And in step 50, the output device 500 converts each of the derived response sentences and emotions into a natural language form, and then combines them to combine and output the response emotions to the spoken emotions and the response sentences to the spoken sentences.

On the other hand, the model update device 600, in step 60, based on the reinforcement learning policy (Policy) learning about the input speech sentences and emotions and the output response sentences and emotions by the policy gradient learning method (Policy gradient training) It is possible to update the dialogue model of the dialogue model building apparatus 100 by performing.

For a large number of collected original data, the learning data is derived by reflecting the accumulated discount value for inducing emotional change, and the conversation model is built by learning based on the set learning technique for the derived learning data. to generate a response sentence and response emotion for each input speech sentence and speech emotion, convert the generated response emotion and response sentence into a natural language form, and then combine and utter to improve the accuracy of the speaker's emotion. and can conduct effective conversations reflecting emotions.

Although the above has been described with reference to preferred embodiments of the present invention, those skilled in the art can variously modify and change the present invention within the scope without departing from the spirit and scope of the present invention described in the claims below. You will understand that you can.

100: dialogue model building device
111: original data collection module
112: discount cumulative compensation value calculation module
113: learning module
114: dialogue model building module
200: receiving device
300: pre-processing device
400: response generator
500: output device
600: model update device

Claims (11)

a dialogue model building device for calculating a cumulative discount compensation value for inducing a change in the speaker's emotions for a plurality of collected original data, and constructing a dialogue model by reflecting the calculated cumulative discount compensation value;
a receiver for receiving sentences and emotions included in utterances;
a pre-processing unit for pre-processing the sentences and emotions received from the receiving unit into one learning data;
a response generating device for deriving response sentences and emotions, respectively, based on a dialogue model built for one learning material of the preprocessor; and
Containing an output device that converts each of the derived response sentences and emotions into a natural language form and then combines them to combine the response emotions for the utterance emotions and the response sentences for the utterance sentences,
The dialogue model building device,
an original data collection module for collecting a plurality of original data;
a discount cumulative compensation value calculation module for calculating a discount cumulative compensation value for each of the original data;
a learning module that generates learning materials by reflecting the accumulated discount value calculated on the respective original materials, and then performs learning on the generated learning materials based on a predetermined learning algorithm; and
Containing a dialogue model building module for building a dialogue model based on the learning result,
The discount cumulative compensation value calculation module,
The discount calculation reward value is initially set to reward=0,
If index+2n+2<2, it is determined whether the emotion of the episode of x[index +2n+2] is happiness, and if it is happy, the current discount calculation compensation value = the previous discount calculation compensation value+r ⁿ ,
After increasing to n = n+1, it is provided to repeatedly perform all original data,
Here, r is the discount rate, x[index+2n+2] is an episode of one original data with sentences and emotions, index is the identification information of the original data, and n is the learning data and the original corrected with the discount cumulative reward value. A dialogue agent system using the speech emotion history, characterized in that it is distance information from the material. delete delete According to claim 1,
The discount cumulative compensation value is
If a sentence with the same emotion is uttered immediately following the response of a random response sentence in the original data, it increases based on the previous discount cumulative reward value,
A dialogue agent system using a speech emotion history, characterized in that when a sentence of the same emotion is uttered after a predetermined number of times as a response to an arbitrary response sentence of the original material, it is reduced based on a previous discount accumulated compensation value. According to claim 1, wherein the dialogue agent system using the speech emotion history
A model update device that updates the dialog model by learning the sentences and emotions included in the utterance and the derived response sentences and emotions with policy gradient training based on a predetermined reinforcement learning policy (Policy) Conversation agent system using speech emotion history, characterized in that it further comprises. a dialog model building step of calculating a cumulative discount reward value for inducing a change in the speaker's emotions with respect to a plurality of collected original data, and constructing a dialogue model by reflecting the calculated cumulative discount compensation value;
a receiving step of receiving sentences and emotions included in the utterance;
a pre-processing step of pre-processing the received sentences and emotions into one learning material;
a response generating step of deriving a response sentence and an emotion based on a dialogue model built for the one learning material, respectively; and
Containing an output step of converting each of the derived response sentences and emotions into a natural language form and then combining them to combine the response emotions for the utterance emotions and the response sentences for the utterance sentences,
The dialogue model building step is,
Collecting a large number of original data,
Calculate the discount cumulative compensation value for each of the original data,
After generating a learning material by reflecting the accumulated discount value calculated on the respective original data, learning is performed on the generated learning data based on a predetermined learning algorithm,
It is provided to build a dialogue model based on the learning results,
The discount cumulative compensation value is,
Initial set reward=0,
If index+2n+2< 2, it is determined whether the emotion of the episode of x[index +2n+2] is happiness, and if it is happy, the current discount calculation reward value = the previous discount calculation compensation value + r ⁿ ,
After increasing to n = n+1, it is provided to repeatedly perform all original data,
Here, r is the discount rate, x[index+2n+2] is an episode of one original data with sentences and emotions, index is the identification information of the original data, and n is the learning data and the original corrected with the discount cumulative reward value. Conversation agent method using speech emotion history, characterized in that it is distance information from material. delete delete The method of claim 6, wherein the discount cumulative compensation value is
If a sentence with the same emotion is uttered immediately following the response of a random response sentence in the original data, it increases based on the previous discount cumulative reward value,
A conversation agent method using a speech emotion history, provided to decrease based on a previous discount cumulative reward value when a sentence with the same emotion is uttered after a predetermined number of times as a response to an arbitrary response sentence of the original material. 7. The method of claim 6, wherein after the outputting step
A model update step of updating the dialog model by learning the speech sentences and emotions input by policy gradient training and the output response sentences and emotions based on a predetermined reinforcement learning policy (Policy) Conversation agent method using speech emotion history further comprising.

delete

Images