JP2020071676A - Speech summary generation apparatus, speech summary generation method, and program - Google Patents

Abstract

To generate a sufficiently shortened and highly accurate summary sentence from a speech voice, while sufficiently reflecting emotion of a speaker during a speech.SOLUTION: A speech voice summary generation apparatus includes: a speaker identification unit which identifies speakers of a speech from speech voice data: a voice separation unit which separates the speech voice data into speech units for each of the speakers identified by the speaker identification unit; a voice recognition unit which recognizes the speech voice data by speech unit separated by the separation unit to generate speech voice texts; a summary sentence generation unit which generates a summary sentence text by summarizing the speech voice texts generated by the voice recognition unit; and an emotion analysis unit which derives emotional expression of each of the speakers by analyzing the speech voice data by speech unit, and outputs the summary sentence text by adding the derived emotional expression to the summary sentence text, replacing a part of the summary sentence text with the emotional expression, or associating the emotional expression with the summary sentence text.SELECTED DRAWING: Figure 3

Description

  The present invention relates to a dialogue summary generation device, a dialogue summary generation method, and a program. More particularly, the present invention provides a summary from recorded dialogue voices available in, for example, a Customer Relationship Management (CRM) system that records and manages telephone or face-to-face dialogues between customers and contacts. A technique for creating and outputting a generated summary.

  Various technologies have been proposed for recording and managing a dialogue voice made between a customer and a business on the business side. In recent years, for the purpose of compliance with business operators, measures against complaints from customers, and evaluation and education of business operators, the content of dialogue is not limited to telephone calls but also includes face-to-face dialogue. Is requested to be recorded.

  As an example, in a call recording system for recording and searching the contents of an operator's call in a call center, which is a department for responding to a call from a customer, generally, a public telephone is installed on the premises of a call center operated by an operator. An exchange (PBX) is installed in which calls and receptions from a public switched telephone network (PSTN) are concentrated, and a voice call is distributed to a fixed telephone in a call center by this exchange.

  Therefore, if a call recording server branching from this exchange is provided, the call can be recorded and stored in a voice data file. On the operator side, a terminal device such as a PC (Personal Computer) may be provided together with a voice response extension telephone. For example, the operator terminal device retrieves customer information by using a customer name announced by a speaker as a key. The function of displaying the past call history of the customer may be provided.

For each voice call between the customer and operator recorded and accumulated in the voice data file in this way, an outline of the call history is recorded and retained for each telephone call, and after the call ends, this history is viewed and used as a report. Output is required. It is desirable to generate a textual summary from the recorded voice call in order to quickly verify and review the contents of this history of responses.
As a technique for creating a summary text from such voice data, there is a technique for converting the voice in a voice data file into a character code by a voice recognition process and generating a summary text from the character-coded voice text data. By generating a textual summary, the contents of the response history can be easily grasped, a list is provided, and the words in the text can be used as keywords to perform a search. It will be possible.

  For example, in Patent Document 1, voices recorded on a recording medium by a video tape recorder (VTR) are voice-recognized and converted into character code strings, and the importance of the constituent elements of the sentence in the voice-recognized character code strings. , Typically, by classifying parts of speech such as nouns, verbs, particles, adjectives, and degrees of importance given to phrases such as nominative cases, objective cases, predicates, etc., by referring to a pre-registered importance table, Disclosed is a technique for automatically generating a summary sentence by combining constituent elements in a sentence determined to have high importance.

  Further, in Patent Document 2, important sections are extracted from speech, topic boundaries are detected using the appearance distribution of the extracted important sections, important sections included in each topic section are semantically classified, and Disclosed is a technique for generating a text-based abstract of a topic from voice.

JP-A-8-212228 JP 2000-284793 A

  However, it is difficult to immediately apply the technology disclosed in each of the above patent documents to, for example, telephone answering work in a call center. Because the voice call between the customer and the operator is usually the acquisition / confirmation of the customer information, the acquisition / confirmation of the inquiry content, the acquisition / confirmation of the reply content to the inquiry, the presentation / confirmation of the customer's understanding level and the exemption content, etc. This is because it is inevitably redundant because it goes through many stages, and as a result of repeating the same utterance content, the dialogue often takes a long time. In addition, call recording data that is recorded and accumulated all day long for many operators makes it difficult to promptly check and check the response history.

  Therefore, even if a well-known abstract sentence creating technique is applied to a voice call text obtained by directly recognizing a voice call, there is an inconvenience that the generated summary sentence is inevitably redundant and long. It was not convenient.

On the other hand, the emotions of the speaker during the dialogue are not uniform. For example, if the utterance of "yes" is voice-recognized during the dialogue, is it the "yes" that the speaker of the utterance consented and uttered, or was the "yes" forced to consent, It can be an utterance based on different emotions.
However, with the conventional technology, it is not possible to reflect the emotion in the utterance of the speaker during the dialogue in the summary.

  The present invention has been made in view of the above problems, and an object thereof is a highly accurate summary sentence that is sufficiently shortened from a dialogue voice and sufficiently reflects the emotion in the utterance of the speaker during the dialogue. It is an object of the present invention to provide a dialogue summary generation device, a dialogue summary generation method and a program capable of generating a dialogue.

  In order to solve the above problems, according to an aspect of the present invention, a speaker identification unit that identifies a speaker of a dialogue from dialogue voice data, and the dialogue voice for each speaker identified by the speaker identification unit. A voice separating unit for separating data into utterance units, a voice recognizing unit for recognizing the dialogue voice data in the utterance units separated by the voice separating unit to generate a dialogue voice text, and the voice recognizing unit. A summary generation unit that summarizes the generated dialogue voice text to generate a summary text, and analyzes the dialogue voice data for each utterance to derive an emotional expression for each speaker, and the derived emotional expression To the summary sentence text, replace part of the summary sentence text with the emotion expression, or output in association with the summary sentence text. Apparatus is provided.

  The emotion analysis unit further analyzes time-series emotional transitions for each speaker in one dialogue by analyzing the conversation voice data in units of the utterance, and the emotions derived for each speaker are analyzed. The transition may be output in association with the summary text.

The dialogue summary generation device further extracts from the dialogue voice text generated by the voice recognition unit, an emotional word indicating an emotion of each speaker, and converts the extracted emotional word into a corresponding emotional expression, A second emotion analysis unit that replaces at least a part of the summary text with the converted emotion expression may be provided.
The dialogue voice generation device further determines whether the same or similar text appears multiple times in the dialogue voice text of one dialogue unit, and when the same or similar text appears multiple times. A redundancy elimination unit may be further provided for deleting a text that appears ahead in time series.
The redundancy eliminating unit further refers to an important word table that defines important words in advance, extracts the text defined in the important word table from the dialogue voice text, and positions the text immediately before the extracted text. And extracting the second text in which the reading of the extracted text at least partially matches the reading, and deleting the searched text from the dialogue voice text.

The dialogue summary generation device further analyzes the dialogue voice text generated by the voice recognition unit to extract a number, assigns different units and weights depending on the type of the extracted number, and generates the summary. A text correction unit may be provided to supply the unit.
The dialogue summary generation device may further include a voice acquisition unit that records a conversation voice or a face-to-face conversation voice to acquire the conversation voice data.

  According to another aspect of the present invention, a step of identifying a speaker of the dialogue from the dialogue voice data, a step of separating the dialogue voice data for each identified speaker into utterance units, and a step of separating the dialogue voice data A step of generating a dialogue voice text by performing voice recognition on the separated utterance unit, a step of summarizing the generated dialogue voice text to generate a summary text, and a step of generating the dialogue voice data on the utterance unit basis. Derives emotional expressions for each speaker by analysis, adds the derived emotional expressions to the summary sentence text, replaces a part of the summary sentence text with the emotion expression, or associates with the summary sentence text. And a step of outputting as a dialogue summary generation method.

  According to still another aspect of the present invention, there is provided a dialogue voice summary generation program for causing a computer to execute a dialogue summary generation process, wherein the program identifies the speaker of the dialogue from the talk voice data. Speaker identification processing, voice separation processing for separating the dialogue voice data into utterance units for each identified speaker, and voice recognition of the dialogue voice data in the separated utterance units to obtain dialogue voice text. A voice recognition process to generate, a summary generation process to summarize the generated dialogue voice text to generate a summary sentence text, and analyze the dialogue voice data for each utterance unit to derive an emotional expression for each speaker. , Adding the derived emotional expression to the summary text or replacing part of the summary text with the emotional expression, or corresponding to the summary text And emotion analysis to only output is intended for causing a process including execution, interactive summarization program is provided.

  According to the dialogue summary generation device, the dialogue summary generation method, and the program according to the present invention, a highly accurate summary sentence that is sufficiently shortened from the dialogue voice and sufficiently reflects the emotion in the utterance of the speaker during the dialogue. Can be generated. Therefore, it contributes to improving the usefulness of the summary of the dialogue voice.

It is a figure which shows an example of the network structure of the audio processing system which concerns on embodiment of this invention. It is a block diagram which shows an example of a functional structure of the voice recognition server which comprises the voice processing system of FIG. FIG. 3 is a block diagram showing an example of a functional configuration of a summary generation server which constitutes the voice processing system of FIG. 1. 3 is a flowchart showing an example of a processing flow of a voice recognition process executed by the voice recognition server in FIG. 2. It is a figure explaining the speaker identification (S2) of FIG. 4 with respect to audio | voice data, and isolation | separation (S3) of an audio | voice into the utterance unit. 4 is a flowchart showing an example of a processing flow of a summary generation process executed by the summary generation server of FIG. 3. 5 is a flowchart showing an example of a detailed processing flow of conversion into natural speech and separation into summary units (S5) in FIG. 4. 8 is a flowchart showing an example of a detailed processing flow of separation (S53) into summary units in FIG. 7. It is a figure which shows an example of the recognition result text | voice recognized by the speech recognition by execution of S4 of FIG. It is a figure which shows an example of the syntax analysis result of the recognition result text of FIG. It is a figure which shows an example of the morphological analysis result of the recognition result text of FIG. FIG. 10 is a diagram showing an example of the recognition result text obtained by separating the recognition result text of FIG. 9 into summary units by performing S5 of FIG. 4. 5 is a flowchart showing an example of a detailed processing flow of a summation analysis (S6) in FIG. 4. FIG. 6 is a diagram illustrating the summation analysis (S6) of FIG. 4 for the voice data of FIG. 5. It is a figure which shows an example of the punctuation mark table applied to recognition result text. It is a figure which shows an example of the unit weight table applied with respect to a recognition result text. It is a figure which shows an example of the unnecessary word table applied with respect to a recognition result text. It is a figure which shows an example of the character substitution table applied to a recognition result text. It is a figure which shows an example of the important word table applied with respect to a recognition result text. It is a figure which shows an example of the affirmative word table applied to a recognition result text. It is a figure which shows an example of the negative word table applied to recognition result text. It is a figure which shows an example of the output of the emotion analysis result with respect to a customer's conversation voice. It is a figure which shows an example of the output of the emotion analysis result with respect to a dialogue voice of an operator. It is a figure which shows an example of the output of the emotion analysis result with respect to several dialogue voices of an operator. It is a figure which shows an example of the emotion word table applied to the recognition result text. It is a figure which shows an example of the summary text to which the emotion analysis result was added. It is a figure which shows the example of application to the summary text of the emotional word table of FIG. It is a figure which shows an example which separated the recognition result text of a voice conversation for every speaker into the summary unit. It is a figure which shows the other example which isolate | separated the recognition result text of a voice conversation for every speaker into the summary unit. It is a figure which shows an example of the abstract sentence produced | generated from the recognition result text of FIG. It is a figure which shows an example of the user interface of the summary display of dialog voice. It is a figure which shows the example of a display of the emotion analysis result which can be displayed with the summary of dialog voice. It is a figure which shows the example of a display of the voice recognition result of a dialog voice, the natural language processing result, and the corresponding summary result. It is a figure which shows an example of the hardware constitutions of each apparatus in this embodiment.

  Hereinafter, embodiments for carrying out the present invention will be described in detail with reference to the accompanying drawings. The embodiment described below is an example as a means for realizing the present invention, and should be appropriately modified or changed depending on the configuration of the device to which the present invention is applied and various conditions. It is not necessarily limited to the embodiment. Further, not all of the combinations of features described in the present embodiment are essential to the solving means of the present invention. The same configurations will be described with the same reference numerals.

<Network configuration of voice processing system of this embodiment>
Hereinafter, an example of recording a call made between the customer and the operator of the call center through the telephone network will be described, but the present embodiment is not limited to this. In this embodiment, for example, instead of a call, a summary sentence can be similarly generated also for a dialogue voice that is recorded by collecting a face-to-face dialogue with a sound collection device such as a microphone.
FIG. 1 is a diagram showing a non-limiting example of a network configuration of a voice processing system according to this embodiment. Referring to FIG. 1, the voice processing system includes a PBX (exchange) 1, a voice acquisition server 2, a call recording server 3, a control server 4, a voice recognition server 5, an emotion analysis server 6, a summary generation server 7, and a dialogue summary. A PC (Personal Computer) 9 that can be used for inquiry is provided. All or a part of the PBX 1, the voice acquisition server 2, the call recording server 3, the control server 4, the voice recognition server 5, the emotion analysis server 6, the summary generation server 7, and the PC 9 are installed in a call center premises, and LAN (Local Area) is used. They may be interconnected by an IP (Internet Protocol) network such as an intranet 8 such as Network / WAN (Wide Area Network).

Alternatively, all or part of the voice acquisition server 2, the call recording server 3, the control server 4, the voice recognition server 5, the emotion analysis server 6, the summary generation server 7, and the PC 9 may be connected to a remote IP connection such as the Internet. It may be installed outside the call center as appropriate.
In particular, when an administrator other than the operator of the call center operates the dialogue summary inquiry PC 9 to perform inquiry or update processing of the dialogue voice summary which is the response history in the summary sentence database, the dialogue summary inquiry PC 9 It is not necessary to be installed in the vicinity of the operator, but it is suitable to be installed outside the call center as appropriate through a remote IP connection.

  The voice processing system may further include another PC 10 that connects to or incorporates a microphone that is connected to the voice processing system via the intranet 8 or the Internet. According to this structure, the face-to-face dialogue voice collected by the microphone of the PC 10 can be input to the voice processing system according to the present embodiment to generate a summary sentence of the face-to-face dialogue voice.

  The PBX 1 accommodates extension telephones in the call center, connects these extension telephones to each other, and connects the telephone terminals 12 of the operators to the PSTN (public telephone network) 13 via the private lines 11a, 11b, 11c. The telephone connection between each operator's telephone terminal 12 and the customer's telephone terminal 14 connected to the PSTN 13 is realized by circuit-switching connection.

The PBX 1 in FIG. 1 is connected to the customer's telephone terminal 14 via a public switched telephone line network such as the PSTN 13, but instead of or in addition to this, by providing an IP network connection function. , A voice over internet protocol (VoIP) network or the like, may be connected to a customer's IP call terminal having an IP telephone function via a voice packet communication network. In this case, the voice acquisition server 2 to be described later is connected to the customer's IP call. A voice call between the terminal and the operator's telephone terminal 12 can be obtained. The customer's telephone terminal 14 may be a fixed telephone, a mobile telephone, or a smartphone.
<Functional configuration of each server device>

  The voice acquisition server 2 is branched and connected to the PBX 1, acquires the call voice between the operator's telephone terminal 12 and the customer's telephone terminal 14, and uses the obtained call voice as an identifier (for example, an extension number) of the operator's telephone terminal 12. ) And supply it to each server. Alternatively, the voice acquisition server 2 may be branched and connected to the line between the terminating device (DSU) of the PSTN 13 and the PBX 1.

Under the control of the control server 4, the call recording server 3 compresses the call voice supplied from the voice acquisition server 2 after an incoming call as necessary, and acquires the obtained voice data, for example, NAS (Network Appliance Storage) or the like. The dialogue voice file (the dialogue voice file 31 in FIG. 2) constituted by the large-scale external storage device is stored and stored in the database.
Preferably, the call recording server 3, when analog voice is supplied from the voice acquisition server 2, samples the analog voice waveform represented by a voltage with a predetermined bit depth and a predetermined sampling frequency to generate a digital voice. And is stored in the dialogue voice file 31.

  This digital voice data may be stored in the dialogue voice file 31 after being compressed. Various known methods can be used at various compression rates for compressing the recorded sound, and as a non-limiting example, recording is performed by monaural 1/5 compression, monaural 1/10 compression, or stereo uncompression. Audio is compressed. Alternatively, the call recording server 3 may store and save the voice data supplied from the voice acquisition server 2 in the call voice file 31 without converting and compressing the voice data.

The call recording server 3 also writes call information acquired as call control information in a call information file (not shown) in association with the call voice data stored in the call voice file 31 for each call voice data. This call information is provided by PBX1.
The call information acquired by the call recording server 3 includes, for example, incoming call start information (including the incoming call start time stamp), outgoing call start information (including the outgoing call start time stamp), call start information (including the call start time stamp), Call control information such as call end information (including call end time stamp), caller telephone number, callee telephone number, caller channel number, caller number, callee channel number, callee telephone number (callee extension number, etc.) ) And other call identification information.

The call information further includes speaker identification information that identifies the polarity of whether the utterance in the recorded call is inbound, that is, from the customer side or outbound, that is, from the operator side. This speaker identification information can be acquired by the PBX 1, and in the case of SIP (Session Initiation Protocol), for example, it can be grasped at the time of session configuration at the time of call generation. Specifically, for example, at the time of session configuration, In the Invite command sent from the calling side to the called side, the IP address and port number used by the calling side for reception are specified in the SDP (Session Description Protocol) that describes the information necessary for starting the session. In response to this, the IP address and port number used by the callee for reception are specified in the SDP in the 200 OK message sent from the callee to the caller, and the specified IP address and port number, respectively. RTP (Realtime Transport P audio data is transmitted and received according to the protocol. Therefore, by obtaining the IP address and port number used by each of the calling side and the called side for reception, the speaker identification information of each utterance in one call can be obtained, and the customer identification information in one call can be obtained. The utterance and the operator's utterance can be distinguished or separated as necessary.
In the case of ISDN, the speaker identification information can be acquired as a physical pin position of a line terminating device (Digital Service Unit: DSU).

  These call information are preferably displayed in real time on these display devices in cooperation with a CTI program running on the control server 4 or the operator's PC 9 that implements the CTI (Computer Telephony Integration) protocol. You can do it.

  The call recording server 3 also includes a customer information database (not shown) in which customer information centering on customers who already have a contact history is pre-registered. This customer information includes personal information that identifies the customer, such as customer name, address, registered customer phone number, date of birth, age group, gender, other customer attributes, product purchase history, and response history. , Can be appropriately output to a terminal device that can be operated by an operator in response to an instruction input by the operator.

  The call recording server 3 may be connected, for example, between the PSTN 13 and the PBX 1 instead of being connected to the premises line 8. With this configuration, the call recording server 3 can be connected to the above-mentioned speaker. The identification information can be obtained directly. Further alternatively, the call recording server 3 may be connected to the premises line 8 and directly acquire the call voice supplied to the premises line 8 without separately installing the voice acquisition server 2.

  The control server 4, based on the data and control information supplied from the voice acquisition server 2, the call recording server 3, the voice recognition server 5, the emotion analysis server 6, and the summary generation server 7, the processes executed by these servers. Controls the sending and receiving of data traffic and control information between servers. Alternatively, the voice recognition server 5 and the summary generation server 7 may access the call voice file 31 and the call information file held by the call recording server 3 and interface with the dialogue summary inquiry PC 9 via the control server 4. You may provide it directly instead. In this case, the voice processing system does not need to include the control server 4 separately.

Under the control of the control server 4, the voice recognition server 5 reads out the conversation voice data accumulated and saved in the conversation voice file 31 for each call from off-hook to on-hook, and outputs the conversation voice for one call in a plurality of utterance units. To separate. This separation into utterance units is to identify a silent section and divide the dialogue voice into the silent sections, which will be described later with reference to FIG.
In the present embodiment, the voice recognition server 5 analyzes the dialogue voice data for each separated utterance unit to extract a feature amount, and uses various recognition dictionaries such as a voice recognition dictionary (the voice recognition dictionary 32 in FIG. 2). By referring to this, a known voice recognition technique is applied to convert the conversational voice data into a character code string, and the converted character code sequence is output to a file as a conversational voice text. In the present embodiment, the dialogue voice text output by the voice recognition server 5 includes a text segmented into summary units (summary unit text in FIG. 2). The process of dividing the dialogue voice text into summary units will be described later with reference to FIGS. 4, 7, and 8.

  The emotion analysis server 6 receives the dialogue voice data supplied from the call recording server 3 as an input, and provides a quantitative index indicating the emotion of the speaker, such as mood, satisfaction, stress, and reliability for each speaker. It is output as a speaker emotion analysis result. The emotion analysis result can be output as a change in each emotion index in a predetermined period such as one call or all day. Details of the emotion analysis processing executed by the emotion analysis server 6 will be described later with reference to FIGS. 6 and 22 to 24.

  The summary generation server 7 reads the conversation voice texts stored in the conversation voice text file 33 and divided into summary units for each call, and executes a summary generation process to convert the generated dialogue summary sentence into a summary sentence. It is output as a text (summary text 38 in FIG. 3). Details of this abstract generation processing will be described later with reference to FIG.

  The summary generation server 7 may read the dialogue voice text of one speaker in one call, for example, the utterance of the operator to generate a summary sentence, and the answer part extracted from the utterance of the other speaker, for example, the customer. (Described later) may be added to the summary sentence, or the summary sentence may be created from the dialogue voice texts of both speakers. In the latter case, it is preferable to associate the speaker identification information with the dialogue voice text.

The summary text generated for each call can be appropriately output to an output device such as a display device such as the PC 9 or a printer device for the dialog summary inquiry in response to the inquiry input. It may be output in association with the call start time, the call end time, and the caller identification information of the call (information for identifying whether the call is received from the customer or the call originated from the operator) decoded from the above.
Preferably, the summary sentence displayed and output on the PC 9 or the like can be appropriately updated by a correction input by the operator. By learning the update result and updating the important word table, unnecessary word table, various conversion tables, etc. that should be referred to when generating the summary, it is possible to generate a more accurate and simple summary. Becomes
In the present embodiment, the abstract generation server 7 further refers to the emotion word table (the emotion word table 37 of FIG. 3) and the like by using the dialogue voice text supplied from the voice recognition server 5 as an input, and the dialogue speech text The emotion expression part is extracted and converted into an emotion expression word that should be included in the summary sentence.

  The network and hardware configurations shown in FIG. 1 are merely non-limiting examples, and each server and database may be integrated as needed, or each component may be connected to external equipment such as an ASP (Application Service Provide). May be installed.

<Functional configuration example of the voice recognition server 5>
FIG. 2 is a diagram showing a non-limiting example of a functional configuration of the voice recognition server 5 according to the present embodiment.
Among the function modules of the voice recognition server 5 shown in FIG. 2, as for functions realized by software, a program for providing the function of each function module is stored in a memory such as a ROM and is read out to the RAM to be executed by the CPU. It is realized by executing. Regarding the function realized by the hardware, for example, by using a predetermined compiler, a dedicated circuit may be automatically generated on the FPGA from a program for realizing the function of each functional module. FPGA is an abbreviation for Field Programmable Gate Array. Further, a Gate Array circuit may be formed in the same manner as the FPGA and realized as hardware. Moreover, you may make it implement | achieve by ASIC (Application Specific Integrated Circuit). Note that the configuration of the functional blocks shown in FIG. 2 is an example, and a plurality of functional blocks may constitute one functional block, or one of the functional blocks may be divided into blocks that perform a plurality of functions. Good. The same applies to the functional configurations of the summary generation server 7 shown in FIG. 3 and other server devices.
Referring to FIG. 2, the voice recognition server 5 includes a voice recognition pre-processing unit 51, a voice recognition unit 52, a voice recognition post-processing unit 53, and an Azai analysis unit 54.

  The voice recognition pre-processing unit 51 reads a conversation voice file for each call from the conversation voice file 31 stored and stored by the call recording server 3, and detects a silent section from the read conversation voice file of one call. The detected silent section is divided into utterance units in the dialogue. The voice recognition preprocessing unit 51 also supplies a plurality of utterance units delimited from the conversation voice file of one call to the voice recognition unit 52 for each utterance unit, and the voice recognition unit 52 performs the voice recognition process in the utterance unit. To run.

  The voice recognition unit 52 receives the dialogue voice for each utterance unit supplied from the voice recognition pre-processing unit 51, executes voice recognition processing, and supplies the dialogue voice text for each utterance unit to the voice recognition post-processing unit 53. .. The voice recognition unit 52 can convert the voice data of the conversation voice into the conversation voice text by referring to the voice recognition dictionary 32 capable of defining the important words or the important sentences to be recognized accurately, for example. The voice recognition unit 52 may be mounted on a known voice recognition engine, while the voice recognition pre-processing unit 51, the voice recognition post-processing unit 53, and the analysis unit 54 may be mounted on the control server 4, for example.

  The speech recognition post-processing unit 53 performs syntax analysis, morphological analysis, and the like on the dialogue voice text for each utterance unit output by the voice recognition unit 52, and divides the dialogue voice text into summary units and divides into summary units. The dialogue voice text 33 is output. The syntactic analysis result and the morphological analysis result may be associated with the call voice text divided into summary units. The summarization unit is a unit of subdivision that is further subdivided as a processing unit of the summarization generation process so that the summarization sentence can be easily and highly accurately generated from the call voice text of the utterance unit. For details, see FIG. And will be described later.

  The speech recognition post-processing unit 53 may refer to the speech recognition dictionary 32 that defines weighting for each important word, and weight the extracted abstract units. For example, date, time, address, telephone number, and the like are often important words to be left in the summary sentence, and the speech recognition post-processing unit 53 weights these words to reduce erroneous conversion.

The Aizuchi analysis unit 54 detects texts that are estimated to be the answers such as “Yes” and “No” from the dialogue voice texts divided by the summary unit supplied by the voice recognition post-processing unit 53 and detected. Determine if the text is a match or answer. Based on this determination result, the azuchi analysis section 54 deletes the text determined to be an azure from the dialogue voice text 33 divided by the summary unit output by the voice recognition post-processing section 53.
On the other hand, the summation analysis unit 54 also includes the text determined to be the answer in the dialogue voice text 33 so that it is included in the summary sentence generated by the summary generation server 7, and at the same time, in the dialogue voice text, "acknowledge". Tag that it is. Details of this summation analysis processing will be described later with reference to FIGS. 13 and 14.

<Example of Functional Configuration of Summary Generation Server 7>
FIG. 3 is a diagram showing a non-limiting example of the functional configuration of the summary generation server 7 according to the present embodiment.
Referring to FIG. 3, the summary generation server 7 includes a text correction unit 71, a redundancy elimination unit 72, a summary sentence generation unit 73, an emotion analysis unit 74, and a summary sentence shortening unit 75.

  The text correction unit 71 reads the dialogue voice text 33 divided into summary units, and corrects and corrects the dialogue voice text in order to facilitate the generation of the summary sentence based on the syntactic analysis result and the morphological analysis result. The dialogue voice text is output to the redundancy elimination unit 72.

  The redundancy elimination unit 72 eliminates the redundancy of the corrected dialogue voice text supplied from the text correction unit 71. Specifically, the redundancy elimination unit 72 deletes unnecessary words, duplicate sentences, and the like from the dialogue voice text by referring to the unnecessary word table 35, for example, and the dialogue speech to be supplied to the summary sentence generation unit 73. Shorten the text. The redundancy elimination unit 72 outputs the shortened dialogue voice text from which the redundancy has been eliminated to the summary sentence generation unit 73.

  The summary sentence generation unit 73 reads the shortened dialogue voice text supplied from the redundancy elimination unit 72, refers to the important word table 34, the unnecessary word table 35, and the various conversion tables 36 to generate the summary sentence text. .. The summary generating unit 73 may generate one summary text for each call. The summary sentence output by the summary sentence generation unit 73 may be, for example, a simple style of a report tone obtained by converting the spoken language of the phonetic voice text, for example, a style of stop speech.

  In the present embodiment, the summary sentence generation unit 73 acquires, from the emotion analysis server 6, a quantitative index indicating the emotion of the speaker in the conversation as the emotion analysis result of the speaker, and analyzes the acquired emotion of the speaker. The results can be included in the summary text to be generated and can be displayed on the display device concurrently with or in association with the summary text. The speaker emotion analysis result supplied from the emotion analysis server 6 includes, for each speaker, a quantitative index such as mood, satisfaction, stress, and reliability.

The emotion analysis unit 74 refers to the emotional word table 37 from the summary sentence generated by the summary sentence generation unit 73, extracts the emotional expression portion in the summary sentence text, and determines the emotional expression word to be included in the summary sentence. The converted emotion expression word is converted to replace the emotion expression part extracted in the summary text.
The summary sentence shortening unit 75 shortens the summary sentence so that when the summary sentence supplied from the summary sentence generating unit 73 exceeds a predetermined length, for example, a threshold of a predetermined number of characters, the summary sentence length is within the threshold. , And outputs the shortened summary text as the summary text 38.

<Processing procedure of speech recognition processing in the speech recognition server 5>
FIG. 4 is a flowchart showing a non-limiting example of the processing procedure of the voice recognition processing executed by each unit of the voice recognition server 5.
In S1, the voice recognition preprocessing unit 51 of the voice recognition server 5 reads the dialogue voice data filed for each call from the dialogue voice file 31.
In S2, the voice recognition preprocessing unit 51 of the voice recognition server 5 identifies the speaker in the dialogue voice read in S1. Specifically, the voice recognition preprocessing unit 51 can identify a speaker, for example, a customer and an operator, from the dialogue voice by referring to the speaker identification information of the call information associated with the dialogue voice file. it can.

Specifically, the voice recognition pre-processing unit 51 refers to a call information database (not shown) to determine the speaker identification information in one call, so that each speaker of the utterances in one call is identified. It is possible to identify whether it is a customer or an operator.
In the voice recognition unit 52 in the latter stage, the dialogue voice data is voice-recognized for each identified speaker, and in the summary sentence generation unit 73 of the summary generation server 7 which generates a summary from the voice-recognized dialogue voice text, By referring to the time stamp of the dialogue recording, the texts of the recognition results of both speakers can be associated with each other.

The voice recognition pre-processing unit 51 converts the dialogue voice data of the utterance identified as one speaker, for example, the utterance of the operator from the dialogue voice data of the other speaker, for example, the utterance identified as the utterance of the customer. The summary generation server 7 may be preferentially supplied. This is based on the finding that the utterance of one speaker, typically, the operator, as a summary sentence generation source, can obtain information sufficient for summarizing the response history more efficiently.
Alternatively, the voice recognition preprocessing unit 51 may voice-recognize only one speaker, for example, only the conversation voice data of the utterance identified as the utterance of the operator, and convert the voice data into the conversation voice text. By limiting the target of speech recognition, the hardware resources in the speech recognition server 5 that performs speech recognition with high load can be reduced, the real-time property of the speech recognition processing and the summary sentence generation processing is improved, and the dialogue speech text file It is also possible to reduce the resource capacity such as.

In S3, the voice recognition preprocessing unit 51 of the voice recognition server 5 separates the conversation voice data, which is read for each call, in which the speaker is separated, into the utterance units, and the conversation voice is separated into the utterance units. Is supplied to the voice recognition unit 52.
Specifically, the voice recognition pre-processing unit 51 detects a certain silent section in the dialogue voice data, and divides the voice into the detected silence sections, thereby cutting out the voiced section to obtain dialogue speech in utterance units. To separate.

As shown in FIG. 5, the conversation voice file for one call is composed of two channels, CH1 and CH2. The voice of CH1 is, for example, the utterance of the customer, and the voice of CH2 is, for example, the utterance of the operator.
The voice recognition preprocessing unit 51 detects a silent section having a fixed length. The silent section to be detected may be, for example, a silent section of 1.5 seconds or more, and its lower limit value may be adjusted, for example, between 1 second and 2 seconds. The lower limit value of this silent section is called a first threshold value. The lower limit value of this silent section can be set, for example, in consideration of the time required for breathing. In addition, it is preferable that the lower limit value of the silent section is set so that, for example, the sound repellent "t" during the utterance of "I told you" is not mistakenly detected as the silent section.

With reference to FIG. 5, the voice recognition preprocessing unit 51 detects and detects a silent section (SL11, SL12, ..., SL16) having a length equal to or longer than a first threshold value from the voice of the customer of CH1. A voiced section (SP11, SP12, ..., S17) between the two silent sections is extracted. Each of the extracted voiced sections (SP11, SP12, ..., S17) becomes one utterance unit in the voice identified as the customer, and in the present embodiment, the voice recognition supplied to the voice recognition unit 52. It becomes a unit. Each voiced section can be considered as a section spoken without breathing.
Similarly, with reference to FIG. 5, the voice recognition preprocessing unit 51 detects a silent section (SL21, SL22, ..., SL26) having the first threshold as the lower limit value from the voice of the operator of CH2. , A voiced section (SP21, SP22, ..., S27) between the two detected silent sections is extracted. Each of the extracted voiced sections (SP21, SP22, ..., S27) becomes one utterance unit in the voice identified as the operator.

Returning to FIG. 4, in S4, the voice recognition unit 52 of the voice recognition server 5 performs voice recognition processing for each identified speaker on the conversational voice data input from the voice recognition preprocessing unit 51 in units of utterances. It executes and outputs the dialogue voice text which is the text-formed dialogue voice.
In the present embodiment, the speech recognition processing is performed on the conversational voice data in units of utterances as described above. The silent section makes it possible to infer that the speaker has switched during the silent section, or that the same speaker has changed the topic or content. Therefore, it can be estimated that the continuity of the utterance content is poor before and after the silent section, and the recognition accuracy can be expected to be improved by recognizing the dialogue voice text by the utterance unit.

A known voice recognition engine can be applied to this voice recognition process.
As an example of conversion into a character code string in the voice recognition process executed by the voice recognition unit 52, for example, a feature amount extracted from a voice waveform in the conversation voice data that has undergone various conversion processes is defined in advance. The voice waveform data can be converted into a character code string by performing a comparison process with the reference acoustic pattern for each phoneme.

  In the voice recognition dictionary 32 referred to by the voice recognition unit 52 and the voice recognition post-processing unit 53, important words (or important sentences) that are supposed to be targets of voice recognition in advance and that include important information to be included in a summary sentence are stored. Since the data is defined, only the phoneme string of the conversational speech data corresponding to the important word defined in the speech recognition dictionary 32 may be extracted and given meaning. In addition, weighting may be given to important words (or important sentences) defined in the voice recognition dictionary 32. Of the dialogue voice data read by the voice recognition unit 52, the dialogue voice data portion corresponding to the defined important word may be converted into dialogue voice text and output as a voice recognition result.

  FIG. 9 shows a non-limiting example of the dialogue voice text which is the speech recognition result of the utterance unit generated from the dialogue voice data, which is output by the voice recognition unit 52 in S4. In the example of Fig. 9, "Dialogue summarization processing consists of deleting unnecessary utterances and expressions as well as conversion from spoken language to written language. It can be selected according to the characteristics of the processing target data." It becomes the utterance unit extracted between two silent intervals. As shown in FIG. 9, the speech recognition result for each utterance output in S4 may include a plurality of sentences that are not separated by punctuation marks or the like as one unit.

Returning to FIG. 4, in S5, the voice recognition post-processing unit 53 of the voice recognition server 5 converts the voice recognition result output by the voice recognition unit 52 in S4 into natural utterances, and also divides it into summary units. The voice recognition post-processing unit 53 can also add a type and weight to the dialogue voice text in the abstract unit divided in S5 based on the result of the syntactic analysis and the morphological analysis.
The details of the conversion process in S5 will be described later with reference to FIGS. 7 and 8.

In S6, the summation analysis unit 54 of the voice recognition server 5 detects a text estimated to be a response such as “yes” or “no” from the dialogue voice texts divided into summary units, and the detected text is Judge whether it is a hammer or an answer.
Based on this determination result, the azuchi analysis section 54 deletes the text determined to be an azure from the dialogue voice text 33 divided by the summary unit output by the voice recognition post-processing section 53. On the other hand, the consonant analysis unit 54 includes the text determined as the answer in the call voice text 33 so that it is included in the summary sentence generated by the summary generation server 7, and at the same time, in the dialogue voice text, “acknowledgement” is given to the text element. Is added (type is added). The details of the summation analysis processing in S6 will be described later with reference to FIGS. 13 and 14.
In step S7, the Azai analysis unit 54 outputs the dialogue voice text divided into the summary units to which the text determined to be the answer is added.

<Detailed Processing Procedure of Speech Recognition Post-Processing in Speech Recognition Post-Processing Unit 53>
FIG. 7 is a flowchart showing an example of a detailed processing procedure of the voice recognition post-processing executed by the voice recognition post-processing unit 53 in S5 of FIG.
Referring to FIG. 7, in S51, the speech recognition post-processing unit 53 of the speech recognition server 5 refers to the speech recognition dictionary 32, and talks in utterance units, which are the speech recognition results output by the speech recognition unit 52 in S4. Performs voice text parsing.
In S52, the voice recognition post-processing unit 53 refers to the voice recognition dictionary 32 and executes the morphological analysis of the dialogue voice text in utterance units. Either of the syntactic analysis in S51 and the morphological analysis in S52 may be executed first, or may be executed simultaneously in parallel.

FIG. 10 shows a non-limiting example of the syntactic analysis result obtained by executing the syntactic analysis process in S51 on the dialogue voice text of the utterance unit shown in FIG. As shown in FIG. 10, in the syntax analysis result output in S51, the relationship between morphemes in the text is structured.
FIG. 11 shows a non-limiting example of a morpheme analysis result obtained by executing the morpheme analysis processing in S52 on the dialogue voice text of the utterance unit shown in FIG. As shown in FIG. 11, the morpheme analysis result may include the written character, the reading, and the acquired part-of-speech classification (large classification, middle classification, small classification) for each extracted morpheme.

Returning to FIG. 7, in S53, the voice recognition post-processing unit 53 subdivides the dialogue voice text in utterance units into summary units based on the analysis results of the syntactic analysis and the morphological analysis in S51 and S52.
FIG. 8 is a flowchart showing an example of a detailed processing procedure of the separation processing into summary units executed by the speech recognition post-processing unit 53 in S53 of FIG.
In S531, the speech recognition post-processing unit 53 determines whether or not the part-of-speech type of the delimiter unit obtained as a result of the morphological analysis and the syntactic analysis is a noun. When the part-of-speech type of the division unit obtained as a result of the analysis is a noun S (S531: Y), the process proceeds to S532. On the other hand, when the part-of-speech type of the delimiter obtained as a result of the analysis is other than a noun, the processing of S532 and subsequent steps is skipped, the processing ends, and the processing proceeds to S6.

  In S532, the speech recognition post-processing unit 53 determines whether the beginning of the group (group) of delimiter units obtained as a result of the morphological analysis and the syntactic analysis is other than a noun. When the head of the group of delimiters is other than a noun (S532: Y), the process of S533 and the subsequent steps is skipped, the process ends, and the process proceeds to S6. On the other hand, if the head of the group of delimiters is a noun (S5323: N), the process proceeds to S533.

  In S533, the speech recognition post-processing unit 53 determines whether the delimiter unit obtained as a result of the morpheme analysis and the syntactic analysis is the noun + α. When the delimiter unit is the noun + α, that is, when a non-noun such as a particle is included at the end (S533: Y), the speech recognition post-processing unit 53 combines the delimiter unit with the immediately preceding delimiter unit in S534. The process ends and the process proceeds to S6. On the other hand, when the delimiter unit is not the noun + α, that is, when it is only the noun (S533: N), the speech recognition post-processing unit 53 combines the delimiter unit with the immediately preceding delimiter unit, and then returns to S532. , S532, and S533, the determination of the delimiter unit is repeated.

In FIG. 12, the speech recognition post-processing unit 43 is output in S5 of FIG. 4 after the dialog speech text of the utterance unit shown in FIG. 9 is input and the syntactic analysis result shown in FIG. 10 and the morphological analysis result shown in FIG. It is an example of a dialogue voice text to be performed.
Square symbols in FIG. 12 indicate delimiters of summary units. As shown in FIG. 12, by performing the conversion into natural utterance and the separation processing into summary units in S5, consecutive "dialogue", "summary", and "processing" are combined into one summary unit, and A series of "processing", "object", and "data" are combined into another summary unit, respectively, to form a summary unit.
The voice recognition post-processing unit 43 of the voice recognition server 5 may further add the type and weight to each of the separated dialogue voice texts in the abstract unit by referring to the voice recognition dictionary 32. In FIG. 12, the summarization unit “dialogue summarization process” and the summarization unit “data to be processed” are weighted as important summarization units to be included in the summarization sentence.

<Detailed Processing Procedure of Ajai Analysis Processing in Azai Analysis Unit 54>
FIG. 13 is a flowchart showing a non-limiting example of a detailed processing procedure of the matching analysis processing executed by the matching analysis unit 54 of the voice recognition server 5 in S6 of FIG.
With reference to FIG. 13, in S61, the consonant analysis unit 54 of the voice recognition server 5 acquires the dialogue voices of both speakers, for example, the customer and the operator, from the dialogue voice file 31. Since the conversation voice file 31 is provided with a time stamp capable of associating both speakers with each other for each call, the Aizuchi analysis unit 44 outputs the conversation voices of both speakers constituting one call unit. Can be obtained. Alternatively, the conversation voices of both speakers may be associated with each other by giving a common identifier to each of the conversation voices of the speakers forming the call unit. In S61, the acquired conversational voices of both speakers and the conversational voice text obtained by voice recognition of the conversational voices are input.

In S62, the Aizuchi analysis unit 54 compares the dialogue voices of both the customer and the operator, and determines whether or not a short utterance can be detected while the other party of the dialogue is uttering.
Referring to FIG. 14A, since the short utterance (SP14) in the conversation voice of the customer of CH1 is a short utterance made during the utterance (SP24) of the operator of CH2 who is the other party of the dialogue, S62. Detected in. The short utterance to be detected in S62 may be, for example, less than 2 seconds.
When a short utterance is not detected while the other party is uttering (S62: N), the processes of S63 to S68 are skipped, the process is terminated, and the process proceeds to S7. On the other hand, when a short utterance is detected while the other party is uttering (S62: Y), the process proceeds to S63.

  In S63, the Azumi analysis section 54 searches the dialogue voice text of the voice recognition result having the same time stamp as the short utterance detected in S62, and whether the voice recognition result of the short utterance can be estimated as an answer or not. That is, it is determined whether or not the answer is a candidate. For example, if the short utterance is a text such as "Yes", "Yes", "No", or "No", it can be determined as a candidate for the answer. For example, this answer / answer candidate may be set in advance in the summation analysis unit 54.

  When the voice recognition result of the short utterance is not a received answer candidate (S63: N), the process proceeds to S64, it is determined that the short utterance is a cooperation, and the short utterance is deleted from the dialogue voice text to be input for the summary generation. In other words, the short utterance determined to be an amulet in S64 is a meaningless adjusent in the creation of the abstract, and is therefore not a source of the abstract sentence generation. On the other hand, when the voice recognition result of the short utterance is the answer candidate (S63: Y), the process proceeds to S65.

In S65, the summation analysis unit 54 further determines whether or not the voice of the other party of the dialogue has a short silent period during the utterance of the short utterance which is the answer candidate detected in S63.
With reference to FIG. 14A, in the utterance of the operator of CH2 corresponding to the short utterance (SP14) in the voice of the customer of CH1, the voice recognition preprocessing unit 41 in S3 of FIG. Since it does not include a silent section of the length of, it is detected as one utterance unit SP24. In S65, a second threshold value smaller than the first threshold value is used to determine whether or not a short silent section can be detected in the voice of the other party of the dialogue. This second threshold has a value smaller than the first threshold, for example 1 second and may be adjusted between 0.5 and 1.5 seconds.

In S65, during the utterance of the short utterance that is the answer candidate, in the utterance unit (voiced section) of the other party of the dialogue, a short silent section having a length of the second threshold or more is detected (S65: Y ), It is determined in S66 that the short utterance is an answer and the process proceeds to S67. On the other hand, during the utterance of the short utterance that is the answer candidate, when a short silent section having a length equal to or greater than the second threshold value is not detected in the utterance unit (voiced section) of the conversation partner (S65: N), Proceeding to S64, the short utterance, which was a response candidate, is determined to be a humor, and is deleted from the dialogue voice text to be input for the summary generation.
In S67, the Atsuuchi analysis unit 54 separates the voice of the other party of the dialogue into two utterance units before and after the short utterance determined as the answer in S66.

  With reference to FIG. 14B, if it is determined that the voice recognition result of the short speech period (SP14) of the customer of CH1 is determined as the answer candidate, the speech period of the operator of CH2 ( In SP24), a silent section (SL24a) that is equal to or larger than the second threshold and smaller than the first threshold can be detected. In this case, the hammer analysis unit 54 separates the operator's utterance section (SP24) before and after the detected silent section (SL24a), and immediately before the silent section (SL24a), the utterance section (SP24a) and the silent section (SP24a). The utterance section (SP24b) immediately after SL24a) is acquired.

  In S68, the summation analysis unit 54 recognizes in S66 the dialogue voice text obtained by performing voice recognition of the utterance section (SP24a) immediately before the short silent section (SL24a) separated in S67. It is determined that the text is a dialogue voice text that should be paired with, and the text of this pair is associated with each other as the text of the answer and answer and the text that identifies the answer that prompted the answer. Is added to the dialogue voice text file 33 in a summary unit.

<Processing Procedure of Summary Generation Processing in Summary Generation Server 7>
FIG. 6 is a flowchart showing a non-limiting example of the processing procedure of the summary generation processing executed by each unit of the summary generation server 7.
With reference to FIG. 6, in S10, the text correction unit 71 of the abstract generation server 7 reads the conversational voice texts of one call unit from the conversational voice texts 33 divided into the summary units.

In S11, the text correction unit 71 corrects the dialogue voice text read in S10. Specifically, as shown in FIG. 12, the text correction unit 71 inserts punctuation marks in the dialogue voice text of one utterance unit delimited by the summarization unit (processing unit of summarization generation), and then the position of the punctuation mark. Insert a line break with.
FIG. 15 shows a non-limiting example of the punctuation mark table referred to by the text correction unit 71. The punctuation mark table in FIG. 15 defines terms in which punctuation marks or punctuation marks should be inserted immediately after. In FIG. 15, “1” indicates insertion of a reading point and “0” indicates insertion of a punctuation mark. The text correction unit 71 refers to the punctuation mark table of FIG. 15 and matches with suffixes from the delimiter of the summary unit, such as “masuga”, “masuka”, “masu”, and “yes” defined in the punctuation mark table. Is searched for, and a punctuation mark or a punctuation mark is inserted immediately after the searched word according to the definition of the punctuation mark table. The text correction unit 71 may search the search words defined in the punctuation mark table in FIG. 15 in order from the one having the largest number of characters.

The text correction unit 71 further searches for a numerical word extracted by the morphological analysis, and analyzes the meaning of the searched numerical value. In generating a summary sentence for a response history, a number is often an important word that is a keyword in the summary. Therefore, the text correction unit 71 analyzes the meaning of the searched number, acquires the type according to the analyzed meaning of the number, and assigns a unit or weight according to the acquired type.
As the meaning of the numerical words, for example, “date”, “time”, “amount of money”, “telephone number”, “number of pieces” and the like can be given, but the number is not limited thereto.

FIG. 16 is a number-word type table that the text correction unit 71 refers to for giving a type, a unit (notation), and a weight to the analyzed element of the number. Referring to FIG. 16, for example, date, time, and amount of money (yen) are given higher weights than the number (pieces) and temperature (degrees).
On the other hand, the text correction unit 71 may determine that it is a misrecognition and delete the number from the dialogue voice text when the dialogue voice text is searched for a number that is not related to the preceding and following words. In addition, the text correction unit 71 may convert the retrieved numerical words into half-width numbers in order to improve the visibility and the clarity in the summary sentence.

Returning to FIG. 6, in S12, the redundancy removing unit 72 of the digest generation server 7 removes the redundancy in the voice-recognized dialogue voice text and outputs a more simplified or simplified dialogue voice text.
Specifically, the redundancy eliminating unit 72 refers to the unnecessary word table 35 and deletes the unnecessary words from the dialogue voice text.
FIG. 17 shows a non-limiting example of the unnecessary word table 35 referred to by the redundancy eliminating section 72. Referring to FIG. 17, unnecessary words table 35 defines interjections such as “Eh” and fixed greetings such as “I am always indebted.” As unnecessary words.

  The redundancy elimination unit 72 further, when a sentence (or a group having a meaning such as a phrase or a word) that describes the same or similar content appears multiple times from the conversation voice text for one call, Duplicate sentences may be removed from the dialogue voice text as appropriate. Preferably, the redundancy eliminating unit 72 appears forward in time series from the start to the end of the call when a sentence or the like describing the same or similar contents is applied a plurality of times in the conversation voice text for one call. You can delete the sentence that appeared and leave the last sentence that appeared. This is because the sentence near the end of the call is more likely to describe the final conclusion of the response. Also, the sentence that appears last can be presumed to be a repeat by the operator, and in this case, it can be expected that the repeated sentence will describe the exact content that should be left in the summary as a response history. ..

The redundancy eliminating unit 72 may further refer to the important word table 34 and delete the stagnation and repetition of the keywords registered in the important word table 34.
For example, it is assumed that the keyword “eVoice” and the reading “Evoice” are registered in the important word table 34 as keywords.
In this case, if the recognition result is "I will visit a good eVoice at 10 o'clock tomorrow.", The redundancy elimination unit 72 searches for a keyword whose reading partially matches from the beginning immediately before the registered keyword, and searches. Delete the specified word. As a result, the stagnation point can be deleted from the dialogue voice text.
Similarly, if the recognition result is "I will ask eVoice to eVoice at 10 o'clock tomorrow", the redundancy elimination section 72 deletes the forward part of the registered keywords as described above. This allows the repeated portion to be deleted from the dialogue voice text.

  Returning to FIG. 6, in S13, the summary sentence generation unit 73 of the summary generation server 7 generates a summary sentence of the response history from the dialogue voice text output by the redundancy elimination unit 72. Specifically, the summary sentence generation unit 73 shapes the dialogue voice text described in the conversational body into a sentence body. Preferably, the summary sentence generation unit 73 shapes the dialogue voice text described in the conversational body into a sentence body that is a word stop.

FIG. 18 shows a non-limiting example of the style conversion table 36 referred to by the abstract sentence generator 73. Referring to FIG. 18, in the style conversion table 36, the words of the source conversational language (such as "I am", "I say", and "I was talking") are listed in the left column. The words in the writing style of the conversion destination ("is", "is", "was saying", etc.) are defined respectively. The summary sentence generation unit 72 searches the dialogue voice text for the word of the conversion source conversational body defined in the style conversion table 36, and the retrieved sentence word is the corresponding sentence defined in the style conversion table 36. Convert to body words. As a result, the polite word in the dialogue voice text is converted into a concise report style writing style.
It should be noted that in the style conversion table 36 of FIG. 19, the word of the conversion destination corresponding to the word "little" of the conversion source is not defined. In this case, the abstract sentence generation unit 72 may delete the conversion source word from the dialogue voice text.

Returning to FIG. 6, in S13, the abstract sentence generation unit 73 further searches for a predefined important word from the dialogue voice text and includes the searched important word in the summary sentence to be output.
19, 20 and 21 each show a non-limiting example of the important word table 34 referred to by the abstract sentence generation unit 73. With reference to FIG. 17, in the important word table 34, the words “contact” and “confirm” are defined as important words. Important words may be defined in the important word table 34 together with variable weights (points). In FIG. 19, the weights “1” are defined for the words “contact” and “confirm”. Further, the user may be provided with another important word table 34 that can be added or deleted so that proper nouns can be appropriately defined.
The abstract sentence generation unit 73 searches the dialogue voice text for important words defined in the important word table 34, weights the searched important words according to the corresponding weights, and includes them in the abstract sentence to be generated.

FIG. 20 shows a non-limiting example of the important word table 34 that defines important words that are positive expressions (“Yes”, “Understood”, “Iiyo”, ok) etc.), and FIG. A non-limiting example of the important word table 34 that defines important words that are expressions (“No”, “No”, “Refuse”, “No approval”, etc.) is shown. The abstract sentence generation unit 73 also refers to these important word tables 34 to search for important words from the dialogue voice text, weights the searched important words according to the corresponding weights, and determines the summary words to be generated. include. The important words as positive or negative expressions included in FIGS. 20 and 21 may be appropriately converted into a writing style (“accept”, “reject”, etc.).
It should be noted that, preferably, the summary sentence generation unit 73 may generate one sentence for each call unit regardless of whether a plurality of sentences are supplied from the redundancy eliminating unit 72 or a single sentence is supplied. A summary sentence may be generated.

Returning to FIG. 6, in S14, if the summary sentence generated by the summary sentence generation unit 73 exceeds the threshold of a predetermined length, for example, a predetermined number of characters, the summary sentence shortening unit 75 of the summary generation server 7 is within the threshold value. Shorten the summary to be the length of the summary.
Preferably, the summary sentence shortening unit 75 glances at the query result display screen where the dialogue summary sentences are displayed in a list in the output field provided for displaying the summary sentence for each call without scrolling the entire summary sentence. Then, the number of characters in the readable range may be set as the threshold. This eliminates the need for an additional operation for confirming the summary and enables quick visual confirmation of the entire summary.

More specifically, the summary sentence shortening unit 75 may shorten the summary sentence by referring to the various important word tables 34, based on the weights (importance points) given to the important words appearing in the summary sentence. ..
As an example, the summary sentence shortening unit 75 divides the dialogue voice text supplied from the redundancy eliminating unit 72 into punctuation marks (“.”), And for each dialogue voice text sentence, the important words appearing in the sentence are divided. It is possible to add importance points and preferentially select a call text sentence for which a high importance is calculated.
The summary sentence shortening unit 75 outputs the shortened summary sentence to the file of the summary sentence text 38.

In S15 of FIG. 6, in the present embodiment, the summary sentence generation unit 73 outputs the pair of texts to which the type of “acknowledgement” has been generated, which is generated by the summation analysis unit 54 of the voice recognition server 5, and which should be output. Added to.
The dialogue analysis text of FIG. 13 executed by the dialogue analysis unit 54 of the voice recognition server 5 causes the dialogue voice text uttered by one speaker (for example, a customer) to be judged as an answer, and immediately before the answer. The dialogue voice text that utters the answer from the other speaker (for example, the operator), prompts the answer, and identifies what the answer is, is paired, and the type of “answer” is added to each question. It is included in the dialogue voice text as an answer-type dialogue.

  The summary sentence generation unit 73 treats the pair of dialogue voice texts to which the type of “acknowledgement” is given as an important word, refers to the various conversion tables 36, converts the sentence into a sentence style for the summary sentence, and then outputs it. Add to the summary. For example, it is assumed that the dialogue voice text to which the type of "acknowledgement" is added is a pair of "Is shipping correct in a few days (operator's question)" and "Yes (customer's answer)". In this case, the abstract sentence generation unit 73 converts the dialogue voice text pair into "accept shipment in a few days" or the like, and outputs the converted text as an important word (important sentence) in the response history. Include in sentence.

  As another example, the dialogue voice text to which the type of “answer” is given is “Is the item you ordered in the dialogue summary eV-Outline OK (operator's question)” and “Yes, please (customer answer ) ”. In this case, the abstract sentence generation unit 73 converts this dialogue voice text pair into “confirm the dialogue summary eV-Outline for the ordered item” and the like, and the converted text as an important word (important sentence) in the response history. Include it in the summary to be output.

  In S16, the emotion analysis unit 74 of the summary generation server 7 executes the emotion analysis process of the conversation speaker based on the conversation voice text. The emotion analysis unit 74 also provides an interface from the summary generation unit 73 to the emotion analysis server 6, causes the emotion analysis server 6 to execute emotion analysis processing, and supplies the execution result of the emotion analysis processing to the summary sentence generation unit 73. You may. Alternatively, the emotion analysis unit 74 may execute the emotion analysis process of the speaker of the dialogue in which the summary sentence should be generated, without separately providing the emotion analysis server 6. In the following, an example of executing the emotion analysis process using the former emotion analysis server 6 will be described.

The emotion analysis process includes a non-verbal emotion analysis process using the conversation voice data and a linguistic emotion analysis process using the conversation voice text that is the voice recognition result.
In the former emotion analysis processing based on the conversation voice data, the emotion analysis server 6 called from the emotion analysis unit 74 receives the conversation voice data supplied from the call recording server 3 as an input, and, for example, for each speaker, Quantitative indicators that quantify the emotion of the speaker such as satisfaction, stress, and reliability are output as the emotion analysis result of the speaker.

In this emotion analysis processing provided by the emotion analysis server 6, the movements of the speaker's brain waves and the movements of the vocal cords are linked, and the human cannot control the emotion in the process of utterance, and the emotion appears in the voice. It is based on the knowledge that. Therefore, the emotion analysis server 6 can quantify the emotion of the speaker from the dialogue voice data without depending on the language of the speaker's utterance.
In the latter emotion analysis process based on the dialogue voice text, the emotion analysis unit 74 of the summary generation server 7 receives the dialogue voice text supplied from the voice recognition server 5 as an input, extracts emotion words in the dialogue voice text, and With reference to the word table 37, it is converted into an emotional expression to be included in the summary sentence.

  FIG. 22 shows a non-limiting output example of the result of the emotion analysis server 6 performing the emotion analysis process on the conversation voice data of one speaker (customer) in one call unit. With reference to FIG. 22, the transition of the customer's emotion during one call of the customer (CS) is output in time series. FIG. 22 shows an example in which an operator convinces a customer during a call in response to a complaint from the customer. In FIG. 22, the emotional indexes of “emotion” and “satisfaction” both increased from the middle stage to the second half, while the emotional index of “stress” decreased from the middle stage to the second half. Therefore, it is possible to read the transition of emotions that the customer's anger and stress are reduced and the dissatisfaction is satisfactorily converted from the middle stage to the latter half of one call unit.

In addition, an index for evaluating the quality of the response of the operator who is the other speaker can be obtained from the customer emotion analysis result illustrated in FIG.
For example, the emotional index of "joy and anger" is negative and "anger" is high from the beginning of the call, but at the end of the call, the emotional index of "joy and anger" is converted to 0 or positive and the tendency of "joy" is increased. In addition, when the emotional index of “satisfaction” is also converted to 0 or positive and shows a tendency of “satisfaction”, the evaluation of the response history of the operator may be “excellent response” indicating excellent response.
However, at the end of a call, for example, if the customer's “credibility” emotional index shows a negative tendency to “distrust”, it can be evaluated that the customer's utterance content is low in reliability. A “customer attention” may be noted to indicate that the customer's remarks need attention.

On the other hand, in the middle of a call, the emotional index of "anger and anger" suddenly turned to a large negative value, and the emotional index of "satisfaction" also turned to a large negative value, and the tendency of "anger" and "dissatisfaction" continued thereafter. Since it can be evaluated that the operator's remark just before the conversion to minus has caused the customer's anger or dissatisfaction, it may be a “response caution” indicating that it is necessary to confirm the response of the operator.
Also in this case, at the end of the call, for example, if the customer's "reliability" emotional index shows a negative tendency to "distrust", it can be evaluated that the reliability of the utterance content of the customer is low. , "Customer attention" indicating that the customer's remarks need attention may be noted.
Moreover, when the above tendency is not shown, it may be set as “reception normal” indicating that the reception is appropriate.

FIG. 23 shows a non-limiting output example of the result of the emotion analysis server 6 executing the emotion analysis process on the conversation voice data of the other speaker (operator) in one call unit. FIG. 23 shows an example in which the operator feels stress during a call with a customer. In FIG. 23, the emotional index of the “stress level” has a numerical value increasing from the beginning to the end of the call, and the emotional transition that the operator's stress is increasing can be read.
In this case, for example, when the stress in this call is higher than the transition of the numerical value of the emotion index of the stress level up to the previous time, the operator's evaluation index is continuously monitored for the operator's stress index. It may be used as "response caution" indicating that it should be done.

FIG. 24 shows a transition of emotions between a plurality of calls (15 times in FIG. 24) within a certain period (one day, one week, etc.). In FIG. 24, the average value of the emotional index of the operator's “stress level” gradually increases as the number of calls increases, and the emotional transition that the operator's stress increases as the number of calls increases Can be read.
In this case, the operator's evaluation index may be "stop receiving", which indicates that the operator's reception should be stopped and hearing should be performed immediately.

  FIG. 25 shows a non-limiting example of the emotion word table 37 referred to by the emotion analysis unit 74 of the summary generation server 7. Referring to FIG. 25, in the emotional word table 37, the emotional word of the conversion source is displayed in the left column (“Is it okay”, “That's fine. Thank you”, “I was disappointed”, “It's okay”, "Something", "Make it loose", etc.), and the emotional expressions ("Shibu consent", "Pleasure consent", "Discouragement", "Anxiety", "Discomfort", etc.) of the conversion destination are displayed in the right column. It is defined. The emotion analysis unit 74 of the summary generation server 7 retrieves the emotion word of the conversion source defined in the emotion word table 37 from the dialogue voice text, and the retrieved emotion word corresponds to the emotion defined in the emotion word table 37. Convert to expression. As a result, the emotional word in the dialogue voice text is converted into a simple emotional expression.

  FIG. 27 shows a non-limiting example in which the emotion analysis unit 74 refers to the emotion word table 37 of FIG. 25 to generate a summary sentence incorporating an emotion expression from the dialogue voice text as the voice recognition result. With reference to FIG. 26, the emotion analysis unit 74 refers to the dialogue voice text “I replaced the device, but it is broken again, I am disappointed” in the upper part of FIG. Convert to the summary sentence of. The summary sentence to be output can include an emotional expression understood from the dialogue voice text as the voice recognition result. The word "disappointment" after conversion indicates the emotional expression of the speaker (customer) and is included in the output summary.

On the other hand, FIG. 26 shows a non-limiting example in which the emotion expression obtained by the emotion analysis server 7 executing the emotion analysis process from the dialogue voice data (voice color) is added to the summary text in parentheses. With reference to FIG. 26, the emotion analysis server 7 executes the emotion analysis processing on the dialogue voice data that is the basis of the dialogue voice text “There is an insect in food” in the upper part of FIG. 26, For example, when the emotional index of “reliability” of the voice data is negative and indicates a tendency of “distrust”, an emotional expression of “customer attention” indicating that the customer's statement needs attention is generated, The emotion analysis unit 74 of the summary generation server 7 is solved and supplied to the summary sentence generation unit 73. The summary sentence generation unit 73 of the summary generation server 7 supplies the “customer attention” supplied from the emotion analysis server 6 to the summary sentence “food mixed with insects” in the lower part of FIG. 26 generated from the dialogue voice text in the upper part of FIG. Is added in brackets.
As described above, by reflecting the emotional expression of the speaker in the generated summary sentence, it becomes possible to easily grasp the situation of the emotional transition of the speaker and automatically extract the problematic call for which countermeasures should be taken.

Returning to FIG. 6, the summary generation unit 73 of the summary generation server 7 uses the emotion analysis result as described above at S17, as shown in FIG. 27, emotions that are more straightforward and categorized from the emotion words in the summary sentence. It is replaced with an expression and added to the summary sentence to be output as shown in FIG.
In S18, the summary sentence generation unit 73 or the summary sentence shortening unit 75 outputs the finally generated summary sentence to the file of the summary sentence text 38.

With reference to FIGS. 28 to 30, an example of the extraction conversion process from the dialogue voice text divided by the summary unit output by the voice recognition server 5 to the generation of the summary sentence to be finally output will be described.
FIG. 28 shows a non-limiting example of the conversation voice text of one call unit which is output by the voice recognition server 5 and is input to the abstract generation server 7. In the dialog voice text of FIG. 28, one dialog unit of dialog voice text is shown in each line for each identified speaker (operator (OP) or customer (CS)). The break of the summary unit indicated by a square is inserted.
FIG. 29 shows a non-limiting example of the summary sentence text intermediately output by the summary sentence generation unit 73 of the summary generation server 7 from the dialogue voice text shown in FIG. As shown in FIG. 29, the text of 6 utterance units (the text of the 3rd, 6th, 9th, 11th, 14th, and 15th utterance units) is extracted from the text of 20 utterance units of FIG. In addition, each of the extracted utterance-based texts has been converted into a more concise text for a summary sentence. By referring to the important word table 34, the unnecessary word table 35, and the various conversion tables 36, the abstract sentence generation unit 73 converts the conversation voice text of one entire call of FIG. 28 into the intermediate abstract text of FIG. 29. ..

  FIG. 30 shows a non-limiting example of the summary sentence text finally output from the summary sentence generating unit 73 to the summary sentence shortening unit 75 from the intermediate sentence text output in FIG. As shown in FIG. 30, five lines of abstract sentences are generated from the texts of the six utterances extracted and converted in FIG. 29, and the end of each abstract sentence is “hope”, “confirmation”, etc. Has been converted to. In particular, the pair of the operator's utterance (question) on the 5th line and the customer's utterance (answer / answer) on the 6th line in FIG. 29 is “please consent to wait a few days because it is created and mailed” in FIG. It is summarized in one summary. The summary sentence generation unit 73 refers to the important word table 34 and the various conversion tables 36 to generate the finally output summary sentence text of FIG. 30 that functions as a response history. The end of the fifth sentence of the summary sentence in FIG. 30 is converted into “pleasant consent” that reflects the emotional expression of the speaker (customer) by applying the emotional analysis processing described above.

FIG. 31 shows a non-limiting example of the user interface output to the result display device or the like as a result of querying the dialogue voice text of FIG. 28. With reference to FIG. 31, the user interface may include the identified speaker 311, the response content 312 for each utterance, a play button 313, and a speaker emotion analysis result icon 314. By selecting the reproduction button 313 corresponding to the desired utterance, the audio file of the utterance is reproduced.
FIG. 32 shows, as the emotion analysis result, the emotion analysis result obtained from the numerical value of the emotion index for each talker in the call unit inquired in FIG. The feeling is "normal" to "slightly high", stress is "none", "somewhat", etc. 31 and 32 may be displayed on the display device so as to be visible at the same time.

FIG. 33 shows a voice recognition result of conversational voice of an utterance unit identified as a speaker for one call unit (recording time of 1.25.716), a natural language processing result by referring to a corresponding user dictionary, and a link of a voice file. , Is a non-limiting display example showing a list of start and end times. As shown in the lower left of FIG. 33, the summary generated for the call unit is displayed, which facilitates cross-reference between each processing result and the summary. The user interface of FIG. 33 may display the voice recognition result and the natural language processing result editable after the voice file is played back so that the user can correct the error.
In addition, the generated summary in the lower left of FIG. 33 shows that in the dialogue, “approved to view Saxa Fund's prospectus on the Internet” was finally accepted. ”, The emotion analysis result obtained from the numerical values of a plurality of emotion indexes may be added in parentheses such as“ acceptance (please consent) ”or“ acceptance (reluctance acceptance) ”. May be replaced with an expression including the emotional analysis result such as "pleasant consent" or "reluctant consent".
According to the present embodiment, it is possible to integrate and output the dialogue recording data, the speech recognition result of the dialogue voice, the natural language processing result, the emotion analysis result, and the generated summary sentence as described above.

(Example of hardware configuration of each device)
FIG. 34 is a diagram showing an example of the hardware configuration of each device in the voice processing system. The voice acquisition server 2, the call recording server 3, the control server 4, the voice recognition server 5, the emotion analysis server 6, the summary generation server 7, and the PCs 9 and 10 include all or part of the hardware components shown in FIG. 34. Each device 100 shown in FIG. 34 may include a CPU 101, a ROM 102, a RAM 103, an external memory 104, an input unit 105, a display unit 106, a communication I / F 107, and a system bus 108.

  The CPU 101 centrally controls the operation of the apparatus, and controls each component (102 to 107) via the system bus 1088. The CPU 101 functions as a processing unit that executes each process such as a voice recognition process, a summary generation process, or an emotion analysis process. The ROM 102 is a non-volatile memory that stores control programs and the like necessary for the CPU 101 to execute processing. The program may be stored in the external memory 104 or a removable storage medium (not shown). The RAM 103 functions as a main memory, a work area, etc. of the CPU 101. Therefore, the CPU 101 loads various programs or the like from the ROM 102 into the RAM 103 when executing the processing, and executes the programs or the like to realize various functional operations.

  The external memory 104 stores, for example, various data and various information necessary when the CPU 101 performs processing using a program. Further, the external memory 104 stores, for example, various data and various information obtained by the CPU 101 performing processing using a program or the like. The input unit 105 includes various input devices such as a keyboard and a tablet. The display unit 106 is, for example, a liquid crystal display or the like. The communication I / F 107 is an interface for communicating with an external device, and includes, for example, a wireless LAN (Wi-Fi) interface or a Bluetooth (registered trademark) interface. The system bus 108 communicatively connects the CPU 101, the ROM 102, the RAM 103, the external memory 104, the input unit 105, the display unit 106, and the communication I / F 107.

As described above, according to the present embodiment, it is possible to generate a highly accurate summary sentence that is sufficiently shortened and sufficiently reflects the emotion in the utterance of the speaker during the dialogue from the dialogue voice. .. Therefore, it contributes to improving the usefulness of the summary of the dialogue voice.
Each of the above-described embodiments can be realized by combining a plurality of them.
The present invention can also be realized by a program that realizes a part of the above-described embodiments or one or more functions. That is, the program is supplied to a system or device via a network or a storage medium, and one or more processors in the computer (or CPU, MPU, etc.) of the system or device can read and execute the program. is there. Further, the program may be recorded in a computer-readable recording medium and provided.
Further, the functions of the embodiments are not limited to being realized by executing the program read by the computer. For example, an operating system (OS) running on a computer may perform some or all of the actual processing based on the instructions of the program, and the processing may realize the functions of the above-described embodiments.

  Although the embodiments of the present invention have been described in detail above, the above embodiments merely show specific examples for carrying out the present invention. The technical scope of the present invention is not limited to the above embodiment. The present invention can be variously modified without departing from the spirit thereof, and these are also included in the technical scope of the present invention.

1 PBX
2 voice acquisition server 3 call recording server 4 control server 5 voice recognition server 6 emotion analysis server 7 summary generation server 8 private line 9, 10 PC
31 Dialogue Speech 32 Speech Recognition Dictionary 33 Summary Unit Text 34 Duplicate Term Table 35 Unnecessary Word Table 36 Conversion Table 37 Emotional Word Table 51 Speech Recognition Pre-Processing Section 52 Speech Recognition Section 53 Speech Recognition Post-Processing Section 54 Aizuchi Analysis Section 71 Text Correction Section 72 Redundancy Exclusion Section 73 Summary Text Generation Section 74 Emotion Analysis Section 75 Summary Text Shortening Section

Claims (9)

A speaker identification unit for identifying the speaker of the dialogue from the dialogue voice data,
For each speaker identified by the speaker identification unit, a voice separation unit that separates the conversation voice data into utterance units,
A voice recognition unit that voice-recognizes the dialogue voice data in the utterance unit separated by the voice separation unit to generate a dialogue voice text;
A summary generation unit that summarizes the dialogue voice text generated by the voice recognition unit to generate a summary sentence text;
Analyzing the dialogue voice data for each utterance to derive an emotional expression for each speaker, adding the derived emotional expression to the summary text, or replacing a part of the summary text with the emotional expression. , Or an emotion analysis unit which outputs the summary text in association with the summary text,
A dialogue abstract generating apparatus comprising: The emotion analysis unit further analyzes time-series emotional transitions for each speaker in one dialogue by analyzing the conversation voice data in units of the utterance, and the emotions derived for each speaker are analyzed. Output the transition in association with the summary text,
The dialogue summary generation device according to claim 1, wherein From the dialogue voice text generated by the voice recognition unit, extract an emotional word indicating an emotion for each speaker, convert the extracted emotional word into a corresponding emotional expression, and in the converted emotional expression, A second emotion analysis unit that replaces at least a part of the summary text;
The dialogue summary generating device according to claim 1 or 2, characterized in that. It is determined whether or not the same or similar text appears multiple times in the dialogue voice text of one dialogue unit. When the same or similar text appears multiple times, they appear forward in time series. A redundancy elimination unit for deleting text,
The dialogue summary generation device according to any one of claims 1 to 3, characterized in that: The redundancy eliminating unit further refers to an important word table that defines important words in advance, extracts the text defined in the important word table from the dialogue voice text, and positions the text immediately before the extracted text. And searching for a second text in which the reading of the extracted text at least partially matches, and deleting the searched text from the dialogue voice text.
The dialogue summary generation device according to claim 4, wherein A text correction unit that analyzes the dialogue voice text generated by the voice recognition unit to extract a number, assigns different units and weights depending on the type of the extracted number, and supplies the unit to the summary generation unit. To prepare further,
The dialogue abstraction generation device according to any one of claims 1 to 5, characterized in that: Further comprising a voice acquisition unit for recording a conversation voice or a face-to-face conversation voice to acquire the conversation voice data.
The dialogue abstraction generation device according to any one of claims 1 to 6, characterized in that. Identifying the speaker of the dialogue from the dialogue voice data,
Separating the dialogue voice data into utterance units for each identified speaker;
Generating a dialogue voice text by recognizing the dialogue voice data in units of the separated utterances;
Generating a summary text by summarizing the generated dialogue voice text,
Analyzing the dialogue voice data for each utterance to derive an emotional expression for each speaker, adding the derived emotional expression to the summary text, or replacing a part of the summary text with the emotional expression. , Or outputting in association with the summary text,
A method for generating a dialogue summary, comprising: A dialogue abstract generation program for causing a computer to execute a dialogue abstract generation process, the program comprising:
Speaker identification processing for identifying the speaker of the dialogue from the dialogue voice data,
Voice separation processing for separating the dialogue voice data into utterance units for each identified speaker;
A voice recognition process of generating voice dialogue text by voice-recognizing the dialogue voice data in the separated utterance units;
A summary generation process for generating a summary sentence text by summarizing the generated dialogue voice text,
Analyzing the dialogue voice data for each utterance to derive an emotional expression for each speaker, adding the derived emotional expression to the summary text, or replacing a part of the summary text with the emotional expression. , Or an emotion analysis process of outputting the summary text in association with each other, and executing a process including:
A dialogue summary generation program characterized by the following.

Images