JP6818916B2 - Summary generator, summary generation method and summary generation program - Google Patents

Description

Embodiments of the present invention relate to a summary generator, a summary generation method, and a summary generation program.

Conventionally, with the improvement of the accuracy of the voice recognition technology, a system utilizing the voice recognition technology for creating a statement record of a meeting or the like has been proposed. In such a situation, there is a technology that supports the preparation of minutes of meetings and the like, which has been troublesome due to manual preparation. For example, there is a technique for preparing a template of minutes, analyzing the minutes, and then creating the minutes according to the minutes template.

JP-A-2015-138103

However, the prior art has a problem that advance preparation for preparing minutes is complicated. Specifically, since the prior art uses a template of the minutes prepared in advance, the preparation in advance for creating the minutes is complicated.

An object to be solved by the present invention is to provide a summarization generator, a summarization generation method, and a summarization generation program capable of reducing the time and effort of advance preparation.

The summary generation device of the embodiment includes a feature extraction unit, a segment candidate generation unit, and a structured estimation unit. The feature extraction unit extracts the feature information of the word included in the text information. The segment candidate generation unit generates segment candidates to be a constituent unit for display based on the extracted feature information. The structured estimation unit estimates structural information including global structural information to local structural information based on the generated segment candidates and an estimation model for structuring.

The figure which shows the system configuration of the summary generation system which concerns on embodiment. The block diagram which shows the hardware configuration of the summary generator which concerns on embodiment. The block diagram which shows the functional structure of the summary generator which concerns on embodiment. The figure which shows the voice recognition processing result which concerns on embodiment. The figure which shows the morphological analysis processing result which concerns on embodiment. The figure which shows the command expression which concerns on embodiment. The figure which shows the property information and the meaning class of the part of speech which concerns on embodiment. The figure which shows the information derived from the system which concerns on embodiment. The figure which shows the result of the segmentation which concerns on embodiment. The figure explaining the candidate extraction of the structured estimation which concerns on embodiment. The figure explaining the candidate of the display format which concerns on embodiment. The flowchart which shows the flow of the process by the feature extraction part which concerns on embodiment. The flowchart which shows the flow of processing by the segment candidate generation part which concerns on embodiment. The flowchart which shows the flow of processing by the structured estimation part which concerns on embodiment. The flowchart which shows the flow of the process by the display format conversion part which concerns on embodiment.

(Embodiment)
FIG. 1 is a diagram showing a system configuration example of the summary generation system 1 according to the embodiment. As shown in FIG. 1, the summary generation system 1 includes a summary generation device 100 and a terminal device 200. In the summary generation system 1, each device can communicate wirelessly or by wire. Further, the summary generation system 1 may include a plurality of terminal devices 200. The summary generation system 1 creates a summary document visualized as text with a format structure from voice data at a meeting or the like.

In the above-described configuration, the terminal device 200 acquires voice data in a conference or the like, and transmits the acquired voice data to the summary generation device 100 via a network. The voice data is acquired from a microphone or the like connected to the terminal device 200. In the conference, one microphone may be used, or a plurality of microphones may be used. Since the conference may be held between different bases, the summary generation system 1 may include a plurality of terminal devices 200. The terminal device 200 is an information device such as a PC (Personal Computer) or a tablet terminal.

The summary generation device 100 acquires voice data from the terminal device 200, detects an explicit summary instruction of the speaker and an expression of a structured request included in the utterance, and estimates an appropriate display unit (segment). Then, the summary generator 100 rearranges each segment according to its contents in response to a termination instruction from the speaker, converts it into various display formats, and outputs the segment. The summary generation device 100 is an information processing device such as a server device.

FIG. 2 is a block diagram showing a hardware configuration example of the summary generator 100 according to the embodiment. As shown in FIG. 2, the summary generator 100 includes a CPU (Central Processing Unit) 12, a ROM (Read Only Memory) 13, a RAM (Random Access Memory) 14, and a communication unit 15. The above parts are connected to each other by the bus 11.

The CPU 12 controls the operation of the entire summary generator 100. The CPU 12 controls the operation of the entire summary generator 100 by executing a program stored in the ROM 13 or the like using the RAM 14 or the like as a work area. The RAM 14 temporarily stores information and the like related to various processes, and is used as a work area for executing a program stored in the ROM 13 and the like. The ROM 13 stores a program for realizing processing by the summary generator 100. The communication unit 15 communicates with an external device such as the terminal device 200 via a wireless or wired network. The hardware configuration shown in FIG. 2 is an example, and in addition to the above, a display unit for outputting processing results, an operation unit for inputting various information, and the like may be included.

FIG. 3 is a block diagram showing a functional configuration example of the summary generator 100 according to the embodiment. As shown in FIG. 3, the summary generator 100 includes a voice recognition unit 110, a feature extraction unit 120, a segment candidate generation unit 130, a structured estimation unit 140, an instruction unit 160, and a display format conversion unit 170. Has. Some or all of the above parts may be realized by software (program) or by a hardware circuit. Further, the summary generator 100 stores the structure estimation model 150 in the ROM 13 or the like.

The voice recognition unit 110 executes voice recognition processing on the voice data. More specifically, the voice recognition unit 110 accepts the input of voice data transmitted from the terminal device 200. Then, the voice recognition unit 110 executes the voice recognition process and generates text information including the character data of each utterance and the information of the time when the utterance is made. FIG. 4 is a diagram showing an example of the voice recognition processing result according to the embodiment. As shown in FIG. 4, the text information which is the voice recognition result includes the character data of each utterance and the information of the time when the utterance is made.

In addition, the voice recognition unit 110 identifies a utterance section, a silent section, and the like as acoustic features of the voice data, and detects the time length of these sections. The voice recognition unit 110 does not have to be included in the summary generation device 100, and is configured to be processed by the feature extraction unit 120 in the subsequent stage based on the execution result of the voice recognition process / acoustic feature extraction process. May be.

The feature extraction unit 120 extracts feature information included in the text information. More specifically, the feature extraction unit 120 applies the morphological analysis process to the text information generated by the voice recognition unit 110. FIG. 5 is a diagram showing an example of the morphological analysis processing result according to the embodiment. As shown in FIG. 5, the morphological analysis process divides the language into the smallest units of meaningful languages. Then, the feature extraction unit 120 identifies the part of speech information included in the text information and performs the semantic class analysis process. If it is a noun, more detailed information (for example, information such as a person's name and a date) based on the property information is extracted as feature information. For example, in the semantic class analysis process, keywords such as the presence / absence of quantitative expression and unit expression, personal name, organization name, event name, and technical term are extracted as feature information. Subsequently, the feature extraction unit 120 determines a logical element for the text information. For example, in the determination of logical elements, it is determined whether or not the text information includes an ordered bulleted expression, a command expression for structuring, and the like. If the text information contains logical elements, the feature extraction unit 120 adds the metadata thereof.

FIG. 6 is a diagram showing an example of a command expression according to the embodiment. As shown in FIG. 6, the command expressions are "conclusion", "TODO (todo)", "terminate", "terminate", and the like. FIG. 7 is a diagram showing an example of property information and a meaning class of part of speech according to the embodiment. As shown in FIG. 7, the property information and the meaning class of the part of speech are “DATE” for “10 days”, “PERSON” for “west exit”, and the like.

After that, the feature extraction unit 120 determines the segment label for the text information. The segment label is a name that indicates the role of the segment (display unit), and is the property information of the semantic class or noun extracted in the previous stage, or the utterance text that does not have such an element, or the command of the structured instruction. It is metadata given according to whether or not (instruction command) is included. For example, a structuring instruction command refers to an instruction command that suggests the start of structuring, and corresponds to "start of bullet points", "table from here", "table format from here", and the like. Further, the feature extraction unit 120 adds the utterance section and the silent section detected by the voice recognition unit 110 as peripheral information.

In addition, information derived from the summary generation system 1 can also be used as the feature information. For example, the identity extraction unit 120 detects the speaker ID based on the logged-in user of the microphone or the connected terminal device 200 as the system-derived information, and works in conjunction with the usage time of the conference room, the scheduler, and the like. If there is detailed conference information such as the conference title that can be referred to, the conference holding time, the participants, the conference information such as the conference room, and the individual speaker information that inputs the voice during the conference, these are acquired as the identity information. I will do it. FIG. 8 is a diagram showing an example of information derived from the system according to the embodiment. As shown in FIG. 8, the information derived from the system is "A" in the case of "speaker ID", "2015/10/23" in the case of "meeting date and time", and the like.

The segment candidate generation unit 130 generates a variation of the candidate of the minimum structural unit for structuring. Candidates for the minimum structural unit in structuring correspond to the sequence of the same character type such as speaker, paragraph, phrase, kanji and katakana, meaning class, word, part of speech, etc., in descending order of granularity. .. More specifically, the segment candidate generation unit 130 reads the text information generated by the voice recognition unit 110 and the feature information extracted by the feature extraction unit 120. Then, the segment candidate generation unit 130 detects the segment label existing in each feature information. For example, in the detection of segment labels, start instructions, end instructions, and other labels that can be used as clues for structuring are detected.

Subsequently, the segment candidate generation unit 130 groups the feature information that has been read and accumulated so far. For example, in grouping, the regular appearance of similar elements and the appearance pattern of different types of feature information are detected, and the unit of such repetition is set to the same group. For example, similar elements refer to those in which repetitions of elements (three elements) such as "date, place, arbitrary text sentence" appear regularly.

Further, if the segment label includes a structuring end instruction, the segment candidate generation unit 130 performs ordering among the feature information grouped up to that point. Examples of ordering include a method in which ordering is defined in advance for the type of feature information and then the ordering is fixedly defined, and in a specific example of extracted feature information, the character length included in each feature ( There are a method of ordering based on the average character length), a method of ordering based on the content of a specific element (semantic class), and the like.

FIG. 9 is a diagram showing an example of the result of segmentation according to the embodiment. As shown in FIG. 9, by segmentation, text information to which command expressions included in the text for each time, property information and meaning class of part of speech, system-derived information, and the like are added is generated.

The structured estimation unit 140 estimates the structural information based on the segment information. More specifically, the structured estimation unit 140 reads the segment information generated by the segment candidate generation unit 130. Then, the structured estimation unit 140 reads the structure estimation model from the structure estimation model 150. The structure estimation model is learned as training data using a format example suitable for display and a result edited and confirmed in the past. Based on such a structure estimation model, the structure estimation unit 140 presents suitable structuring candidates in order by giving combinations and patterns of appearances of feature information. In the initial presentation of structural information, the most likely structured result is presented from the ordered segment pattern.

Subsequently, the structured estimation unit 140 receives a confirmation instruction from the user. The instruction from the user is received via the instruction unit 160. For example, if there is no problem with the current presentation candidate, the presentation candidate is presented as a fixed structured result. On the other hand, when the confirmation instruction from the user cannot be obtained (when the request for presenting the next candidate is accepted), the following structured result is presented. When presenting the following structured result, not only the combination of segments may be changed, but also the variation may be changed and presented by going back to the method of extracting the segment itself. The presentation of the structuring result may be output from the terminal device 200 or the summary generation device 100.

FIG. 10 is a diagram illustrating an example of extracting candidates for structured estimation according to the embodiment. As shown in FIG. 10, based on the segment information, the structuring result is from the information of the global structure to the information of the local structure.

The display format conversion unit 170 converts the finalized structured result into a display format for the user to view. More specifically, the display format conversion unit 170 reads the structuring result determined by the structuring estimation unit 140. Then, the display format conversion unit 170 reads the display format conversion model. The display format conversion model describes a definition pattern of what kind of display format is presented according to the structured result, and is described by CSS (Cascading Style Sheets), XSLT (XSL Transformations), or the like. It is possible to do.

Subsequently, the display format conversion unit 170 presents the initial conversion result from the structured result and the display format conversion model. In response to this presentation, when a confirmation instruction from the user is received via the instruction unit 160, the conversion result is output as a summary document. On the other hand, when the confirmation instruction from the user cannot be obtained (when the request for presenting the next candidate is accepted), the conversion result having the next highest likelihood is presented. The presentation of the conversion result may be output from the terminal device 200 or the summary generation device 100.

FIG. 11 is a diagram illustrating a candidate display format according to the embodiment. As shown in FIG. 11, when presenting with information of any structure among the structured results, they are displayed in chronological order, displayed for each person's name, or only items and people are displayed.

FIG. 12 is a flowchart showing an example of a processing flow by the feature extraction unit 120 according to the embodiment. As shown in FIG. 12, the feature extraction unit 120 acquires the text information generated by the voice recognition unit 110 (step S101). Then, the feature extraction unit 120 applies the morphological analysis process to the acquired text information (step S102). Subsequently, the feature extraction unit 120 identifies the part of speech information included in the text information and executes the semantic class analysis process (step S103). After that, the feature extraction unit 120 determines the logical element with respect to the text information (step S104).

Then, the feature extraction unit 120 determines the segment label for the text information (step S105). Subsequently, the feature extraction unit 120 adds the utterance section and the silent section detected by the voice recognition unit 110 as peripheral information (step S106). After that, the feature extraction unit 120 detects the speaker ID based on the logged-in user of the microphone or the terminal device 200 as the information derived from the system (step S107). Then, the feature extraction unit 120 detects the detailed conference information and the like managed by the external device (step S108).

FIG. 13 is a flowchart showing an example of a processing flow by the segment candidate generation unit 130 according to the embodiment. As shown in FIG. 13, the segment candidate generation unit 130 reads the text information generated by the voice recognition unit 110 and the feature information extracted by the feature extraction unit 120 (step S201). Then, the segment candidate generation unit 130 reads the segment label included in each feature information (step S202). Subsequently, the segment candidate generation unit 130 groups the feature information accumulated by the reading so far (step S203). After that, the segment candidate generation unit 130 orders the feature information grouped at this point (step S204).

Then, the segment candidate generation unit 130 determines whether or not the segment label includes a structuring end instruction (step S205). At this time, the segment candidate generation unit 130 performs ordering among the grouped feature information (step S206) when the segment label includes a structuring end instruction (step S205: Yes). On the other hand, the segment candidate generation unit 130 returns to the process of step S201 when the segment label does not include the structuring end instruction (step S205: No).

FIG. 14 is a flowchart showing an example of a processing flow by the structured estimation unit 140 according to the embodiment. As shown in FIG. 14, the structured estimation unit 140 reads the ordered segment information generated by the segment candidate generation unit 130 (step S301). Then, the structured estimation unit 140 reads the structure estimation model from the structure estimation model 150 (step S302). Subsequently, the structured estimation unit 140 presents initial structural information (structuring candidates) by giving combinations and patterns of appearances of feature information based on the read segment information and the structural estimation model. Step S303).

After that, when the structured estimation unit 140 receives a confirmation instruction from the user in response to the presentation of the structural information (step S304: Yes), the structured estimation unit 140 assigns the presented candidate as the confirmed structural information (step S305). Further, the structured estimation unit 140 has a high score in the runner-up when the confirmation instruction from the user is not accepted (accepts the request for presenting the next candidate) in response to the presentation of the structural information (step S304: No). Candidates for structural information are presented (step S306). After the candidate is presented, the process returns to step S304, and the user waits for a confirmation instruction.

FIG. 15 is a flowchart showing an example of a processing flow by the display format conversion unit 170 according to the embodiment. As shown in FIG. 15, the display format conversion unit 170 reads the structuring result determined by the structuring estimation unit 140 (step S401). Then, the display format conversion unit 170 reads the display format conversion model (step S402). Subsequently, the display format conversion unit 170 presents the initial conversion result from the structured result and the display format conversion model (step S403).

After that, when the display format conversion unit 170 receives a confirmation instruction from the user in response to the presentation of the conversion result (step S404: Yes), the display format conversion unit 170 outputs the conversion result as a summary document (step S405). In addition, when the confirmation instruction from the user is not accepted in response to the presentation of the conversion result (the request for the presentation of the next candidate is accepted) (step S404: No), the candidate for the conversion result having the highest score is presented. (Step S406). After the candidate is presented, the process returns to step S404, and the user waits for a confirmation instruction.

According to the embodiment, the segment is estimated from the voice recognition processing result for the voice data based on the explicit instruction by the speaker and the expression for the structured request, and the segment is rearranged according to the content, and various types are used. Since it is converted into a display format and presented, the time and effort required for advance preparation can be reduced.

Further, the summary generation device 100 according to the above embodiment can be realized by using, for example, a general-purpose computer device as basic hardware. The program to be executed has a modular structure including each of the above-mentioned functions. Further, even if the program to be executed is recorded on a computer-readable recording medium such as a CD-ROM, CD-R, or DVD in an installable or executable format file and provided, the program is stored in the ROM or the like. It may be incorporated in advance and provided.

Moreover, the above-described embodiment is presented as an example, and is not intended to limit the scope of the invention. These novel embodiments can be implemented in various other embodiments, and various omissions, replacements, and changes can be made without departing from the gist of the invention. In addition, each embodiment can be appropriately combined as long as the contents do not contradict each other. In addition, each embodiment and its modification are included in the scope and gist of the invention, and are also included in the scope of the invention described in the claims and the equivalent scope thereof.

100 Summary generator 110 Speech recognition unit 120 Feature extraction unit 130 Segment candidate generation unit 140 Structured estimation unit 150 Structure estimation model 160 Indicator unit 170 Display format conversion unit

Claims (9)

A feature extraction unit that extracts feature information of words contained in text information,
A segment candidate generation unit that generates segment candidates that are constituent units for display based on the extracted feature information.
A structured estimation unit that estimates a plurality of structural information that is the result of structuring the text information into different structures based on the generated segment candidates and an estimation model for structuring.
An output control unit that outputs the designated structural information among the plurality of structural information as a summary of the text information.
Summary generator with. It further has a voice recognition unit that executes voice recognition processing of voice data and generates the text information.
The summary generator according to claim 1, wherein the feature extraction unit extracts feature information of words included in the generated text information. The voice recognition unit generates the text information in which the character information of the utterance and the time when the utterance is spoken are associated with each other, and extracts the acoustic characteristics of the voice data.
The summary generator according to claim 2, wherein the feature extraction unit extracts the time for the character information and the acoustic feature as peripheral information for the word. The output control unit converts the instructed structural information into a display format for viewing and outputs the information.
The summary generator according to any one of claims 1 to 3. The structured estimation unit presents structural information having a higher likelihood for the estimation model than other structural information based on the estimation model, which is learning data based on past processing results.
When the confirmation instruction is received for the presented structural information, the output control unit outputs the presented structural information as a summary of the text information.
The summary generator according to any one of claims 1 to 4. The summary generator according to claim 4, wherein the output control unit changes the display format according to a user instruction. The structured estimation unit presents a plurality of the structural information in the order of likelihood with respect to the estimation model.
When the confirmation instruction is received for the presented structural information, the output control unit outputs the presented structural information as a summary of the text information.
The summary generator according to claim 1. A summary generation method performed by a summary generator,
Steps to extract the feature information of words contained in text information,
Based on the extracted feature information, a step of generating a segment candidate as a constituent unit for display, and
A structured estimation step that estimates a plurality of structural information that is the result of structuring the text information into different structures based on the generated segment candidates and an estimation model for structuring.
An output control step that outputs the instructed structural information as a summary of the text information among the plurality of structural information.
Summary generation method including. In the summary generator,
Steps to extract the feature information of words contained in text information,
Based on the extracted feature information, a step of generating a segment candidate as a constituent unit for display, and
A structured estimation step that estimates a plurality of structural information that is the result of structuring the text information into different structures based on the generated segment candidates and an estimation model for structuring.
An output control step that outputs the instructed structural information as a summary of the text information among the plurality of structural information.
Summary generator to execute.

Images