JP2013219495A - Emotion-expressing animation face display system, method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an emotion-expressing animation face display system, method, and program allowing for efficient monitoring of telephone correspondence by plural operators.SOLUTION: The emotion-expressing animation face display system comprises: emotion estimation means which estimates the emotion of each of users who are speaking with each of operators at least with voice, on the basis of the voice of each user used for the speech; animation face synthesis means which synthesizes an animation face expressing an estimated emotion for each user on the basis of the emotion estimated by the emotion estimation means; and display means which arranges and displays the animation face of each user synthesized by the animation face synthesis means such that the animation faces can visually be checked together.

Description

  The present invention relates to an emotion expression animation face display system, method and program, and more particularly to an emotion expression animation face display system, method and program for a call center.

  In order to respond to telephone inquiries and orders from customers (also referred to as “clients” or “users”), a call center is set up. Usually, in a call center, a system for a call center is constructed and used so that a plurality of operators can efficiently handle calls from clients. The call center system includes, for example, a telephone exchange, a plurality of operator terminals, a manager (also referred to as “supervisor”) terminal, and the like. In many call center systems, the supervisor monitors the communication between the client and the operator through call monitoring, confirms the state of the call, and has a function for sending advice to the operator as necessary.

JP 2006-330958 A JP 2003-248841 A

  In the above-described conventional call center system, supervisor monitoring of the operator's telephone response is performed by voice monitoring, and only one operator's telephone response can be monitored at a time for one supervisor. There was a problem that it was not right.

  Also, unless there is a request for help from the operator, it is not possible for the supervisor to determine whether the supervisor's advice is necessary for calls that are not being monitored at that time, resulting in delays in response. There was a problem.

  Accordingly, an object of the present invention is to provide an emotion expression animated face display system, method, and program that can solve the above-described problems and can efficiently monitor telephone correspondences of a plurality of operators.

  In order to achieve the above object, the system of the present invention provides a plurality of users, each of which communicates the feelings of a plurality of users who are talking with each of a plurality of operators using at least voice. Emotion estimation means for estimating based on the user's voice used in the game, and animation face synthesis for synthesizing an animation face expressing the estimated emotion for each user based on the emotion estimated by the emotion estimation means Means and display means for displaying the animation faces synthesized by the animation face synthesis means side by side so that they can be visually confirmed together.

  In addition, the method of the present invention provides the voices of the users who are the users of the plurality of users, each of whom is talking with each of the plurality of operators using at least the voice. An emotion estimation step based on the emotion, an animation face synthesis step for synthesizing an animation face expressing the estimated emotion for each user based on the emotion estimated by the emotion estimation step, and the animation face And a display step for displaying the animation faces for each user synthesized in the synthesis step side by side so that they can be visually confirmed together.

  In addition, the program of the present invention can use the emotions of a plurality of users who are communicating with each of a plurality of operators using a voice at least in the computer. An emotion estimation procedure for estimating based on the user's voice; an animation face synthesis procedure for synthesizing an animation face expressing the estimated emotion for each user based on the emotion estimated by the emotion estimation procedure; A display procedure for displaying the animation faces synthesized for each user by the animation face synthesis procedure side by side so as to be visually confirmed together is executed.

  With the above configuration of the present invention, it is possible to efficiently monitor the telephone correspondence of a plurality of operators at a call center at a time.

1 shows a configuration example of a system to which the present invention is applied. 1 is a block diagram illustrating a configuration of an entire system according to a first embodiment of this invention. It is a block diagram which shows the structure of the call center system in the 1st Embodiment of this invention. The animation faces generated by the animation face generation unit are illustrated, and the animation faces of (a), (b), and (c) represent emotions of “joy”, “anger”, and “sorrow”, respectively. . It is a block diagram which shows the structure of the supervisor terminal of the 1st Embodiment of this invention. It is an example of a display in the display apparatus of a supervisor. It is a block diagram which shows the structure of the whole system of the 2nd Embodiment of this invention. It is a block diagram which shows the structure of the operator terminal in the 2nd Embodiment of this invention. It is a block diagram which shows the structure of the supervisor terminal of the 2nd Embodiment of this invention.

DESCRIPTION OF EMBODIMENTS Hereinafter, embodiments for carrying out the present invention will be described in detail with reference to the drawings.
FIG. 1 shows a configuration example of a system to which the present invention is applied. This system includes an exchange (not shown), a call center system 101, a client terminal 105, an operator terminal 107, and a supervisor terminal 109. A plurality of client terminals 105, operator terminals 107, and supervisor terminals 109 may exist. Each of the client terminal 105 and the supervisor terminal 109 includes a SIP (Session Initiation Protocol) control unit 121, an RTP (Real-time Transport Protocol) control unit 123, and an interface unit 125. The interface unit 125 has an interface function with, for example, a microphone, a speaker, a function button, a mouse, a keyboard, and the like, and the supervisor terminal 109 has an interface function with a display device that displays an image in addition to them. . Each of the client terminal 105 and the supervisor terminal 109 may include a microphone, a speaker, a function button, a mouse, a keyboard, a display, and the like.

  In the system of FIG. 1, the client terminal 105 exchanges voice RTP packets with the call center system 101 through the gateway 103. The call center system 101 is connected to the operator terminal 107 and the supervisor terminal 109 by SIP, and each of the terminals 107 and 109 has a function of exchanging SIP by the SIP control unit 121. The RTP control unit 123 provided in the operator terminal 107 encodes the voice input from the microphone via the interface unit 125, transmits it as a voice RTP packet to the target address, and decodes the sent packet. And has a function of sending to the speaker via the interface unit 125. The RTP control unit 123 provided in the supervisor terminal 109 has a function of decoding a video RTP packet sent from the call center system 101 and sending it to a video display (not shown) via the interface unit 125.

  In the system of FIG. 1, the voice uttered by the client is input from the microphone to the client terminal 105 and encoded into a voice RTP packet. This voice RTP packet is sent to the call center system 101 via the gateway 103. The call center system 101 sends the voice RTP packet sent from the client terminal 105 to the operator terminal 107 of the operator in charge. In the operator terminal 107, the voice RTP packet sent from the call center system 101 is decoded and delivered from the speaker to the operator's ear via the interface unit 125. The voice uttered by the operator follows the reverse route and is delivered to the client's ear from the speaker connected to the client terminal 105. In this way, the client and the operator make a call.

  As will be described in detail later, the call center system 101 analyzes the voice of the client using voice RTP packets from the client to estimate emotions, and synthesizes an animated face image representing the estimated emotions. Is encoded into an RTP packet and sent to the supervisor terminal 109. In the supervisor terminal 109, the video RTP packet received from the call center system 101 is decoded by the RTP control unit 123 and displayed on the display unit via the interface unit 125. The supervisor can grasp the call status of the client and the operator from the displayed image, and can give appropriate advice if necessary.

  The supervisor terminal 109 may request the call center system 101 to generate an animation face from the client's voice RTP packet and start receiving the video RTP packet from the call center system 101.

  In this way, the client and the operator start a voice call through the call center system 101, and during that time, the supervisor confirms the animated face generated from the voice state of the client and grasps the status of the client.

  Next, a first embodiment of the present invention will be described in detail with reference to FIGS.

  FIG. 2 shows the overall configuration of the first embodiment of the present invention. The first embodiment of the present invention includes a call center system 101, first, second,..., Nth client terminals (105-1, 105-2,..., 105-N), The first, second,..., Nth operator terminals (107-1, 107-2,..., 107-N) and the supervisor terminal 109 are included. In FIG. 2, the first client terminal 105-1 and the first operator terminal 107-1, the second client terminal 105-2 and the second operator terminal 107-2,..., The Nth client terminal 105-N and the Nth The operator terminal 107-N is configured to be able to make a call via the call center 101. The combination of the client terminal and the operator terminal may be fixed or may be changed as appropriate. The call center system 101 includes animation face generation units (229-1) corresponding to the first, second,..., Nth client terminals (105-1, 105-2,..., 105-N), respectively. , 229-2,..., 229-N). The animation face generation unit 229 estimates the client's emotion based on the client's voice data sent from the corresponding client terminal 105, generates an animation face expressing the emotion, and sends the generated animation face data to the supervisor terminal 109. send.

  In FIG. 2, the client voice data sent from each client terminal and the operator voice data sent from the partner operator terminal are added to each other and sent to the selection unit 245. Based on the selection signal sent from the supervisor terminal 109, the selection unit 245 selects the client's desired voice and the call voice of the partner operator and sends it to the supervisor terminal 109.

  FIG. 3 shows the configuration of the call center system 101 according to the first embodiment of the present invention. The call center system 101 includes an audio data transmission / reception unit 221, a decoder 223, an encoder 225, and an audio data reception / transmission unit 227. The voice data transmission / reception unit 221 receives a voice packet transmitted from the client terminal 105. The voice data transmitting / receiving unit 221 transmits the voice data sent from the operator terminal 107 to the client terminal 105 after the encoder 225 encodes the voice data. The decoder 223 decodes the voice packet received from the client terminal. The encoder 225 converts the decoded audio data into a predetermined protocol or format. The voice data receiving / sending unit 227 sends the voice packet generated by the encoder 225 and accepts voice data from the operator terminal 107.

  The call center system 101 further includes an animation face generation unit 229, an encoder 237, and a video data transmission unit 239. The animation face generation unit 229 includes an emotion estimation unit 231, an animation face synthesis database 233, and an animation face synthesis unit 235.

  The emotion estimation unit 231 estimates client emotions from the client audio data decoded by the decoder 223. For example, it is estimated whether the client's voice is angry or laughing, for example, according to a program prepared in advance or the like.

  The animation face synthesis database 233 is a database for generating an animation face, and is an image file manually uploaded to the system by a user or a system administrator, a file prepared in advance in the system, a past video conference file, or the like. You may register and use a file that stores video as a database.

  The animation face synthesis unit 235 synthesizes an animation face based on the estimation result from the emotion estimation unit 231 and the animation face synthesis data stored in the animation face synthesis database 233. For example, if you are currently estimated to be angry, generate an angry animation during that time, and if you are estimated to be laughing, generate a laughing animation depending on the size of the laugh To do.

  The animation face obtained by the synthesis by the animation face synthesis unit 235 is encoded into a video RTP packet by the encoder 237 and sent from the video data sending unit 239 to the supervisor terminal 109.

  As described above, the voice packet sent from the call center system 101 is sent to the operator terminal 107, and the video packet is sent to the supervisor terminal 109, decoded by the RTP control unit 123 of each terminal, and can be heard or seen. It is played with.

  When the supervisor wants to know the status of multiple clients, multiple video data created by the animation face generation process (see FIGS. 2 and 3) for each client is sent to the same IP address that designates the supervisor. become.

  Next, the emotion estimation unit 231 will be described. In the estimation of emotions by the emotion estimation unit 231, various methods such as a method based on sound intensity analysis of a client's voice and a method based on phoneme analysis can be adopted.

  First, an emotion estimation method based on sound intensity analysis will be described. In this method, a voice analysis unit (not shown) performs voice analysis on the client voice data sent from the decoder 223, and in particular, emotions are estimated from information related to sound intensity (or power). The emotion can be specified by, for example, the type of emotion and the strength of the emotion. The type of emotion may be, for example, “joy”, “anger”, “sorrow”, or the like. The intensity of emotion may be classified into, for example, “strong”, “medium”, “weak”, and the like. In the voice analysis, for example, the voice data is separated into time series for each predetermined frame, and the power deviation between these frames, the average value of the power difference and / or the deviation of the power difference are obtained, and the sound intensity is calculated from the analysis result. Information related to emotions such as the degree and the pattern may be extracted, and the type and intensity of emotion may be estimated based on the information.

  For example, the emotion estimation unit 231 may store the voice analysis result sent from the voice analysis unit (not shown) for a predetermined period and estimate the emotion using the stored analysis data. For example, if the strength of the voice data for three periods of the predetermined reference period is “strong / medium / strong”, “joy”, if “strong / strong / strong”, “anger”, “weak / medium / weak” "If so, prepare a template for" Sorrow. " Of the stored analysis data, for the data for three reference periods, the intensity pattern is compared with the template, and the matching is examined to estimate the emotion at that time.

  As an alternative method, the sum of absolute values of differences in intensity values (Hilbert distance) and the sum of squares of differences in intensity (Euclidean distance) are calculated for the stored audio data for three reference periods, You may judge the near thing as the emotion at that time.

  Next, an emotion estimation method based on phoneme analysis will be described. In this method, a keyword representing emotion is held as a dictionary template, and the emotion is estimated by matching the result of phoneme analysis with the dictionary template. For example, in the case of the emotion of “anger”, a word representing anger (for example, “anger”, “talk”) is prepared as a dictionary template. Dictionary templates are similarly prepared for emotions such as “joy” and “sorrow”. Then, the phoneme data obtained from the phoneme analysis result is compared with these dictionary templates to check the matching, thereby estimating the emotion.

  A method based on sound intensity analysis and a method based on phoneme analysis may be combined. For example, if both estimate the same emotion, the emotion may be selected, and if different, either emotion may be selected probabilistically by a random number.

  As another emotion estimation method, there is also a method for estimating a speaker's emotion based on a feature amount related to the frequency and amplitude of a voice signal. For example, the maximum value of the fundamental frequency and the maximum value of the amplitude of the voice being spoken can be used as the feature amount. The emotion estimation unit 231 estimates the speaker's emotion by comparing the reference data of the feature amount acquired in advance for each emotion with the feature amount data acquired by the speech analysis unit.

  Emotion estimation results (such as emotion type and intensity) obtained by any of the above-described various methods in the emotion estimation unit 231 are sent to the animation face synthesis unit 235.

  Next, the animation face synthesis database 233 will be described. The animation face synthesis database 233 stores, for example, face shape data based on CG characters and face shape data based on photographs. This face shape data is composed of parts such as eyes, nose, mouth, eyebrows, ears, and hair.

  The animation face synthesis database 233 may store facial expression data, facial expression motion data, facial expression pattern data, etc. in addition to facial shape data. The expression data is expression data including a texture of a laughing expression, a texture of a crying expression, a texture in the middle of the expression, and the like in a composition method for pasting textures. The facial expression pattern data is data relating to transition from one facial expression data to another facial expression data, and includes information on facial expression data that can be transitioned from certain facial expression data, information on the transition probability, and the like. The expression motion data is data for generating a facial expression of the CG character. Specifically, when transforming the shape of the face, the time series data of the amount of movement of the vertex coordinates corresponding to the end points such as the eyebrows, eyes, and mouth that generate the facial expression of the facial shape data is the facial expression action data. is there.

  Next, the animation face synthesis unit 235 will be described. The animation face synthesis unit 235 synthesizes an animation face in which the estimated emotion is expressed based on the client's emotion during the call estimated by the emotion estimation unit 231. At the time of composition, data stored in the animation composition database 233 may be used. In order to generate an animation face expressing various emotions, for example, a so-called facial animation technique can be used. Specifically, an animated face expressing an emotion may be generated by changing the shape of the face or pasting the texture of the face. In the method of deforming the shape of the face, for example, each emotion is expressed by deforming the shape of the eyebrows, eyes, mouth, nose, ears, face, etc. based on the expression motion data. In the method of changing the texture of the face, the expression pattern data can be taken into account and the texture can be changed using the expression data.

  FIG. 4 illustrates an animation face generated by the animation face generation unit 229 of the present invention. The animation faces in FIGS. 4A, 4B, and 4C express emotions of “joy”, “anger”, and “sorrow”, respectively. The animation face synthesis unit 235 selects a color expressing the estimated emotion for each user based on the emotion estimated by the emotion estimation unit 231 and adds the selected color to the animation face as a background color. May be. For example, if the client's voice is calm, the animation background is blue, and if the client's voice is angry, the animation background is red. Good.

  FIG. 5 shows the configuration of the supervisor terminal in the first embodiment of the present invention. The supervisor terminal 109 includes a screen composition unit 301, a display unit 303, a selection input reception unit 305, a selection signal output unit 307, an audio processing unit 309, and an audio output unit 311. The screen synthesis unit 301 receives the video data of the animation faces of the respective clients sent from the animation face generation units 229-1, ..., 229-N of the call center system 101, and synthesizes the screen. The synthesized screen data is sent to the display unit 303, and the synthesized screen is displayed on the display device. FIG. 6 shows an example in which an animated face is displayed on the display device. Animated faces that express the emotions of the 12 clients who are talking are displayed side by side in 4 columns and 3 columns. In the display example of FIG. 6, a background color (in this case, red) different from the background color of other emotional faces is added to the background of the angry animated face, making it easier for the supervisor to understand.

  By displaying the list side by side in this way, the supervisor can grasp the state of emotions of a plurality of clients at a time in real time. The supervisor can efficiently send advice to the operator, for example, by starting voice monitoring by giving priority to a client who can determine that the client's anger is strong and the degree of urgency is high.

  For example, when the animated face of the kth client (k is an arbitrary number from 1 to N) has an angry expression, the supervisor can be configured to monitor the call between the kth client and the kth operator. Also good. When monitoring a call, first, the supervisor selects a client to be monitored by an appropriate input means such as a mouse, keyboard, touch panel, or button. The selection input is received by the selection input receiving unit 305, and a selection signal indicating the selection target is sent from the selection signal output unit 307 to the selection unit 245 (FIG. 2) of the call center system 101. In the selection unit 245, the call voice data of the client and the partner operator indicated by the selection signal are selected and sent to the voice processing unit 309 of the supervisor terminal 109. In FIG. 5, since the kth client is selected, the voice data (see FIG. 2) obtained by adding the voice data of the kth client and the voice data of the kth operator is sent to the voice processing unit 309. The voice processing unit 309 performs processing such as decoding, and the voices of the kth client and the kth operator are output from the speaker via the voice output unit 311. In this way, the supervisor can monitor the call between any client and its counterpart operator.

  As described above, in the first embodiment of the present invention, in the call center system, an emotion estimation function for estimating an emotion based on voice input from the client side, and an animation image based on the estimation result are generated. This function is provided for each client, and the generated animation is displayed on a monitor monitored by the supervisor, so that the supervisor can grasp the state of the client at a glance and can efficiently instruct the operator.

  The emotion estimation unit 231 may analyze the voice of a client that needs to generate an animation based on information previously set in the server, and estimate the emotion from the analysis result. In addition, an image that is a base for generating an animation face in accordance with the estimation result may be based on a person's photograph in order to give a real feeling, or may be based on an image such as a character.

  Next, a second embodiment of the present invention will be described in detail with reference to FIGS.

  FIG. 7 shows the overall configuration of the second embodiment of the present invention. Compared with the first embodiment of the present invention shown in FIG. 2, the call center system 101 does not have an animation generation function, and terminals (107-1, 107-2,... -N) are provided with animation face generation units (229-1, 229-2,..., 229-N), respectively, and a client call voice selection unit provided in the call center system 101. 245 is provided as a selection unit 315 (FIG. 9) in the supervisor terminal 109, and the other configurations are the same.

  FIG. 8 shows the configuration of the operator terminal in the second embodiment of the present invention. Compared to the configuration of the call center system 101 in FIG. 3, the operator terminal 107 in FIG. 8 encodes the data in which the synthesized animation face is encoded into video packets by the encoder 237 and the call voice of the client and the partner operator. Thus, the data encoded in the voice packet is multiplexed by the multiplexing / sending unit 239A and sent to the supervisor terminal 109, and the other structures are the same.

  Next, the operation of the operator terminal 107 according to the second embodiment of the present invention will be described with reference to FIG. The voice sent from the call center system 101 to the operator's terminal 107 is received by the voice data transmission / reception unit 221 and decoded by the decoder 223. The audio data decoded by the decoder 223 is duplicated and sent separately to the encoder 225 and the emotion estimation unit 231. The emotion estimation unit 231 estimates emotions such as whether the input voice is currently angry or laughing, and sends the estimation result to the animation face synthesis unit 235. The animation face synthesis unit 235 generates an animation face using various data stored in the animation face synthesis DB 233 based on the estimation result sent from the emotion estimation unit 231. The generated animation face is encoded into video packet data by the encoder 237 and sent to the multiplexing / sending unit 239A. Also, voice packet data obtained by encoding the voice of the call between the client and the operator of the client into voice packets by the encoder 225 is sent to the multiplexing / sending unit 239A. In the multiplexing / sending unit 239A, these video packet data and audio packet data are multiplexed and sent to the supervisor terminal 109.

  By configuring the operator terminal 107 in this way, it is not necessary to go through a call center system having an animation generation function, and the supervisor can grasp the situation by animation even in the case of a peer-to-peer client-operator exchange. .

  FIG. 9 shows a configuration of a supervisor terminal according to the second embodiment of the present invention. The supervisor terminal 109 in FIG. 9 is different from the supervisor terminal in FIG. 5 in that the selection signal output unit 307 is not provided, a demultiplexing unit 313 and a selection unit 315 are added, and the other configurations are the same. From each operator terminal 107, a signal obtained by multiplexing the voice data and animation face data of each client is sent to the demultiplexing unit 313. The demultiplexing unit 313 separates the signal from each operator terminal into animation face data and client voice data. In this case, the voice data of the client means voice data obtained by adding the voice of the partner operator to the voice of the client. The face data of each client is sent to the screen composition unit 301 to compose the screen, and the composition screen is displayed on the display device via the display unit 303. The call voices of the respective clients and operators separated by the demultiplexing unit 313 are sent to the selection unit 315. In the selection unit 315, the call voice of the client (in this case, the k-th client) corresponding to the selection input received by the selection input reception unit 305 and the operator of the other party is selected and sent to the voice processing unit 309. The voice processing unit 309 performs processing such as decoding, and outputs voices of the kth client and the kth operator from the microphone via the voice output unit 311.

  The first effect obtained by the present invention is that a supervisor can simultaneously grasp a client's emotions such as emotions with respect to a plurality of clients in real time.

  The reason is that multiple animated faces created based on each client's voice are displayed side-by-side on a single screen, so it is possible to grasp each operator's response status at a glance without monitoring the call. This is because it can. In supervisor call monitoring in a normal call center, each operator's response is monitored through voice, and one supervisor cannot simultaneously monitor multiple operators, so the situation can be judged efficiently. I could not.

  The second effect is that even if the client is gradually getting angry without the operator's knowledge or suddenly getting angry, the supervisor can grasp the situation and take early action. It is.

  The reason is that it becomes possible for the supervisor to judge the situation of the client based on a moving animation. As a result, the supervisor can grasp the situation without relying only on the judgment of the operator.

The emotion expression animated face display system described above can be realized by hardware, software, or a combination thereof. Also, the emotion expression animation face display method performed by the emotion expression animation face display system can be realized by hardware, software, or a combination thereof. Here, “realized by software” means realized by a computer reading and executing a program.

  The program may be stored using various types of non-transitory computer readable media and supplied to the computer. Non-transitory computer readable media include various types of tangible storage media. Examples of non-transitory computer readable media include magnetic recording media (for example, flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (for example, magneto-optical disks), CD-ROMs (Read Only Memory), CD- R, CD-R / W, semiconductor memory (for example, mask ROM, PROM (Programmable ROM), EPROM (Erasable PROM), flash ROM, RAM (random access memory)). The program may also be supplied to the computer by various types of transitory computer readable media. Examples of transitory computer readable media include electrical signals, optical signals, and electromagnetic waves. The temporary computer-readable medium can supply the program to the computer via a wired communication path such as an electric wire and an optical fiber, or a wireless communication path.

A part or all of the above-described embodiment can be described as in the following supplementary notes, but is not limited thereto.
(Appendix 1)
Emotion estimation means for estimating the emotions of a plurality of users, each of which communicates with each of a plurality of operators using at least voice based on the voices of the users used in the call When,
Based on the emotion estimated by the emotion estimation unit, for each user, an animation face synthesis unit that synthesizes an animation face expressing the estimated emotion;
Display means for displaying the animation faces for each user synthesized by the animation face synthesizing means side by side so that they can be visually confirmed together;
An emotion expression animated face display system characterized by comprising:

(Appendix 2)
An emotion expression animated face display system according to appendix 1,
The animation face synthesizing unit synthesizes the animation face expressing the estimated emotion by synthesizing the data stored in the animation synthesis database based on the emotion synthesized by the emotion estimation unit. Emotion expression animated face display system characterized by doing.

(Appendix 3)
The emotion expression animated face display system according to appendix 1 or 2,
The animation face synthesizing unit selects a color expressing the estimated emotion for each user based on the emotion estimated by the emotion estimation unit, and adds the selected color to the animation face as a background color Emotion expression animated face display system characterized by doing.

(Appendix 4)
The emotion expression animated face display system according to any one of appendices 1 to 3,
The emotion estimation means and the animation face synthesis means are included in a call center system,
The emotion expression animated face display system, wherein the display means is included in a terminal used by a supervisor.

(Appendix 5)
The emotion expression animated face display system according to any one of appendices 1 to 3,
The emotion estimation means and the animation face synthesis means are included in a terminal used by the operator,
The emotion expression animated face display system, wherein the display means is included in a terminal used by a supervisor.

(Appendix 6)
The animation face display system according to any one of appendices 1 to 6,
The emotion expression animation face display system, wherein the terminal used by the supervisor further comprises monitor means for monitoring and outputting a call between the operator selected by the supervisor and the other user.

(Appendix 7)
Emotion estimation step of estimating the emotions of a plurality of users, each of which is talking with each of a plurality of operators using at least voice based on the voices of the users used in the call When,
Based on the emotion estimated by the emotion estimation step, for each user, an animation face synthesis step for synthesizing an animation face expressing the estimated emotion;
A display step for displaying the animation faces for each user synthesized in the animation face synthesis step side by side so that they can be visually confirmed together;
An emotion expression animated face display method characterized by comprising:

(Appendix 8)
The emotion expression animated face display method according to appendix 7,
The animation face synthesis step combines the data stored in the animation synthesis database based on the emotion estimated by the emotion estimation step, thereby synthesizing the animation face expressing the estimated emotion. An emotion expression animated face display method characterized by:

(Appendix 9)
The emotion expression animated face display method according to appendix 7 or 8,
The animation face synthesis step selects a color expressing the estimated emotion for each user based on the emotion estimated by the emotion estimation step, and adds the selected color to the animation face as a background color An emotion expression animated face display method characterized by:

(Appendix 10)
The emotion expression animated face display method according to any one of appendices 7 to 9,
The emotion estimation step and the animation face synthesis step are executed by a call center system,
The emotion expression animated face display method, wherein the display step is executed by a terminal used by a supervisor.

(Appendix 11)
The emotion expression animated face display method according to any one of appendices 7 to 9,
The emotion estimation step and the animation face synthesis step are executed by a terminal used by the operator,
The emotion expression animated face display method, wherein the display step is executed by a terminal used by a supervisor.

(Appendix 12)
The animation face display method according to any one of appendices 7 to 11,
An emotion expression animated face display method characterized by further comprising a monitor step of monitoring and outputting a call between an operator selected by the supervisor and a user of the other party at a terminal used by the supervisor.

(Appendix 13)
On the computer,
Emotion estimation procedure for estimating the emotions of a plurality of users who are communicating with each of a plurality of operators using at least voice based on the voices of the users used in the call When,
Based on the emotion estimated by the emotion estimation procedure, for each user, an animation face synthesis procedure for synthesizing an animated face expressing the estimated emotion;
A display procedure for displaying the animation faces for each user synthesized by the animation face synthesis procedure side by side so that they can be visually confirmed together;
Emotion expression animation face display program characterized by running.

(Appendix 14)
An emotion expression animated face display program according to attachment 13,
The animation face synthesis procedure combines the data stored in the animation synthesis database based on the emotion estimated by the emotion estimation procedure, thereby synthesizing the animation face expressing the estimated emotion. Emotion expression animated face display program characterized by doing.

(Appendix 15)
An emotion expression animated face display program according to appendix 13 or 14,
The animation face synthesis procedure selects a color expressing the estimated emotion for each user based on the emotion estimated by the emotion estimation procedure, and adds the selected color to the animation face as a background color Emotion expression animated face display program characterized by doing.

(Appendix 16)
The emotion expression animation face display program according to any one of appendices 13 to 15,
The emotion estimation procedure and the animation face synthesis procedure are executed in a call center system,
An emotion expression animated face display program characterized in that the display procedure is executed in a terminal used by a supervisor.

(Appendix 17)
The emotion expression animation face display program according to any one of appendices 13 to 15,
The emotion estimation procedure and the animation face synthesis procedure are executed in a terminal used by the operator,
An emotion expression animated face display program characterized in that the display procedure is executed in a terminal used by a supervisor.

(Appendix 18)
The animation face display program according to any one of appendices 13 to 17,
An emotion expression animated face display system, further comprising a monitor procedure for monitoring and outputting a call between an operator selected by a supervisor and a user of the other party at a terminal used by the supervisor.

  The present invention can be used for a call center, a video phone, a video conference system, and various applications used for them.

101 Call Center System 103 Gateway 105 Client Terminal 107 Operator Terminal 109 Supervisor Terminal 121 SIP Control Unit 123 RTP Control Unit 125 Interface Unit 221 Audio Data Transmit / Receive Unit 223 Decoder 225 Encoder 227 Audio Data Reception / Transmission Unit 229 Animation Face Generation Unit 231 Emotion Estimation Unit 233 Animation face synthesis database 235 Animation face synthesis unit 237 Encoder 239 Video data transmission unit 239A Multiplex / transmission unit 245 Selection unit 301 Screen synthesis unit 303 Display unit 305 Selection input reception unit 307 Selection signal output unit 309 Audio processing unit 311 Audio Output unit 313 Demultiplexing unit

Claims (10)

Emotion estimation means for estimating the emotions of a plurality of users, each of which communicates with each of a plurality of operators using at least voice based on the voices of the users used in the call When,
Based on the emotion estimated by the emotion estimation unit, for each user, an animation face synthesis unit that synthesizes an animation face expressing the estimated emotion;
Display means for displaying the animation faces for each user synthesized by the animation face synthesizing means side by side so that they can be visually confirmed together;
An emotion expression animated face display system characterized by comprising: The emotion expression animated face display system according to claim 1,
The animation face synthesizing unit synthesizes the data stored in the animation synthesis database based on the emotion estimated by the emotion estimation unit, thereby synthesizing the animation face expressing the estimated emotion. Emotion expression animated face display system characterized by doing. The emotion expression animated face display system according to claim 1 or 2,
The animation face synthesizing unit selects a color expressing the estimated emotion for each user based on the emotion estimated by the emotion estimation unit, and adds the selected color to the animation face as a background color Emotion expression animated face display system characterized by doing. The emotion expression animated face display system according to any one of claims 1 to 3,
The emotion estimation means and the animation face synthesis means are included in a call center system,
The emotion expression animated face display system, wherein the display means is included in a terminal used by a supervisor. The emotion expression animated face display system according to any one of claims 1 to 3,
The emotion estimation means and the animation face synthesis means are included in a terminal used by the operator,
The emotion expression animated face display system, wherein the display means is included in a terminal used by a supervisor. The animation face display system according to any one of claims 1 to 5,
The emotion expression animation face display system, wherein the terminal used by the supervisor further comprises monitor means for monitoring and outputting a call between the operator selected by the supervisor and the other user. Emotion estimation step of estimating the emotions of a plurality of users, each of which is talking with each of a plurality of operators using at least voice based on the voices of the users used in the call When,
Based on the emotion estimated by the emotion estimation step, for each user, an animation face synthesis step for synthesizing an animation face expressing the estimated emotion;
A display step for displaying the animation faces for each user synthesized in the animation face synthesis step side by side so that they can be visually confirmed together;
An emotion expression animated face display method characterized by comprising: The emotion expression animated face display method according to claim 7,
The animation face synthesis step combines the data stored in the animation synthesis database based on the emotion estimated by the emotion estimation step, thereby synthesizing the animation face expressing the estimated emotion. An emotion expression animated face display method characterized by: The emotion expression animated face display method according to claim 7 or 8,
The animation face synthesis step selects a color expressing the estimated emotion for each user based on the emotion estimated by the emotion estimation step, and adds the selected color to the animation face as a background color An emotion expression animated face display method characterized by: On the computer,
Emotion estimation procedure for estimating the emotions of a plurality of users who are communicating with each of a plurality of operators using at least voice based on the voices of the users used in the call When,
Based on the emotion estimated by the emotion estimation procedure, for each user, an animation face synthesis procedure for synthesizing an animated face expressing the estimated emotion;
A display procedure for displaying the animation faces for each user synthesized by the animation face synthesis procedure side by side so that they can be visually confirmed together;
Emotion expression animation face display program characterized by running.

Images