JP5171501B2 - Server, system, method and program for extracting important words - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a device for extracting a significant word in a conversation, and for presenting it as text data. <P>SOLUTION: A server 10 receives the voice data of a conversation, and separates the received voice data for every speaker, converts each of the separated voice data into text data, selects the prescribed number of text data from among the converted text data from the order of a new speech time, and extracts a significant word, based on an index related with the appearance frequency of each word included in the selected text data. <P>COPYRIGHT: (C)2010,JPO&amp;INPIT

Description

The present invention, server to extract the important word in the conversation, system, and Methods us and program.

  Conventionally, audio data for meetings, conversations, etc. will not be recorded unless it is recorded, and the recorded audio data cannot be known unless it is played back continuously. And searching for information took a long time. Therefore, voice data has been converted to text data using voice recognition technology.

  At this time, if there are a plurality of speakers in the voice data, the speakers cannot be distinguished when converted into text data. Therefore, a method of separating voice data for each speaker has been proposed (see, for example, Patent Document 1). With these techniques, it is possible to record and present voice data as text data for each speaker.

On the other hand, it has been proposed to convert voice data such as a meeting or conversation into text data, extract a keyword from the text data, and distribute an advertisement corresponding to the keyword. For example, Patent Document 2 proposes that in a videophone system, a keyword is extracted from text data and an advertisement corresponding to the keyword is displayed on a display screen. Japanese Patent Application Laid-Open No. 2004-228620 proposes collecting sound in a vehicle and performing voice recognition processing, and distributing an advertisement corresponding to a keyword that matches the recognized vocabulary to the vehicle-mounted device.
Japanese Patent No. 3364487 JP 2002-165193 A JP 2004-226070 A

  However, when speech data is converted to text data by the technique of Patent Document 1, all speech is converted into data, so even important speech and less important speech that is out of the topic are converted to equal data. It had been. As a result, it is difficult to accurately determine the contents of a meeting or conversation, and Patent Documents 2 and 3 do not have a clear description for extracting important keywords. Further, in the techniques of Patent Documents 2 and 3, since the advertisement is distributed to the site where the voice is collected, it cannot be said that the advertisement is distributed to the general public. Furthermore, it is a service corresponding to a keyword extracted from text data, and service information is displayed regardless of the importance of the keyword. Therefore, it cannot be said that the advertisement is effectively distributed.

  Therefore, an object of the present invention is to provide an apparatus capable of extracting an important word in a conversation and presenting it as text data and effectively distributing an advertisement.

  The present invention provides the following solutions.

(1) A server that extracts important words in a conversation,
Receiving means for receiving voice data of the conversation;
Separating means for separating voice data received by the receiving means for each speaker;
Conversion means for converting each of the audio data separated by the separation means into text data;
A selection means for selecting a predetermined number of text data from the newest speech time among the text data converted by the conversion means;
A server comprising: extraction means for extracting an important word based on an index relating to an appearance frequency of each word included in the text data selected by the selection means.

  According to such a configuration, the server receives the voice data of the conversation, separates the received voice data for each speaker, converts each of the separated voice data into text data, and is converted. Among the text data, a predetermined number of text data is selected from those having a new utterance time, and important words are extracted based on an index relating to the appearance frequency of each word included in the selected text data.

  As a result, the server extracts important words from the latest predetermined number of utterance contents in the received voice data. As a result, it is possible to present to the user timely important words representing the content that is currently being discussed from conversations whose content changes every moment.

  As a result, since the user can check the current topic, it is easy to make a comment along this topic. In addition, it is possible to prevent the content of the discussion from diverging by monitoring the presentation of important words that are out of the agenda at a meeting or the like.

(2) a specifying means for specifying the terminal device in response to a processing start request from the terminal device connected via the network;
Connection means for establishing a data communication connection with the terminal device specified by the specifying means;
The server according to (1), further comprising: a transmission unit that transmits the important word extracted by the extraction unit to the terminal device that has established a data communication connection by the connection unit.

  According to such a configuration, the server identifies the terminal device in response to a processing start request from the terminal device connected via the network, and establishes a data communication connection with the identified terminal device. The extracted important words are transmitted to the terminal device with which the data communication connection is established.

  Thus, the server can notify the terminal device of important words that change every moment in a timely manner based on the voice data collected by the terminal device connected via the network. .

(3) Real-time delivery means for delivering the video and audio of the conversation in real time to a terminal device connected via a network ;
Advertisement acquisition means for acquiring advertisement information in accordance with the important words extracted by the extraction means;
The server according to (1), further comprising: an advertisement distribution unit that distributes the advertisement information acquired by the advertisement acquisition unit to the terminal device in conjunction with the real-time distribution.

  According to such a configuration, the server distributes the video and audio of the conversation in real time to the terminal device, acquires the advertisement information corresponding to the extracted important word, and links the acquired advertisement information to the real-time distribution. And distributed to the terminal device.

  As a result, the server can distribute advertisements according to the conversation video and voice to the terminal device in conjunction with the real-time distribution of the conversation video and voice. Can be delivered. Further, since the video and audio of the conversation are harmonized with the distributed advertisement, it is possible to realize an advertisement display without a sense of incongruity, and it is easy to remain in the impression of the user, so a high advertising effect can be expected. Moreover, when the user of the terminal device is a fan of a performer who appears in the video to be distributed, it is possible to expect a situation where the probability of clicking on the advertisement display is high.

  (4) The extraction means uses, as the index, a TF value indicating the frequency of occurrence of each word in the text data and an inverse number of a DF value relating to the frequency of occurrence of each word in the predetermined number of text data. The server according to any one of (1) to (3), wherein a product of a certain IDF value is calculated.

  According to such a configuration, the server considers the passage of time and selects a predetermined number of utterances, so that the product of TF (Term Frequency) and IDF (Inverse Document Frequency) within the predetermined number range. TF / IDF values based on the above are calculated.

  Thus, the server can extract an important word based on the appearance frequency by using the TF / IDF technology considering the freshness of the phrase.

  (5) When the selection unit detects that the text data includes a predetermined type of word, the selection unit selects text data after the predetermined type of word is spoken (1). The server according to any one of (4) to (4).

  According to such a configuration, when the server detects a predetermined type of phrase, for example, a topic conversion word such as “By the way” or “Now”, it is important to target the remarks after the detected phrase. Extract words.

  As a result, the server extracts important words for a period in which they are spoken on the same topic, so that it is possible to reduce the possibility that the current important word is overlooked due to a mixture of different topics.

(6) A synonym database for determining synonyms of each word included in the text data is further provided.
The server according to any one of (1) to (5), wherein the extraction unit calculates the index related to the appearance frequency of each word including the synonyms stored in the synonym database.

  According to such a configuration, the server further includes a synonym database for discriminating synonyms of each word included in the text data, and includes the synonym stored in the synonym database, and an index related to the appearance frequency of each word. Is calculated.

  Thus, since the server includes a synonym database, synonyms can be regarded as the same phrase and important words can be extracted. As a result, it is possible to absorb the fluctuation of expression due to different speakers.

  (7) The extraction unit is configured to determine the index of each word included in the text data of the speaker based on the number of text data with the same speaker in the text data selected by the selection unit. The server according to any one of (1) to (6), wherein weighting is performed on the server.

  According to such a configuration, when there are a plurality of utterances with the same speaker, the server weights each word included in the utterance of the speaker according to the number of utterances. Thus, the server can increase the importance of words included in the content of the speech of a speaker who has a large number of statements.

(8) The conversion means associates volume data indicating the volume of the audio data with the text data,
(1) to (7), wherein the extraction unit weights the index of each word included in the text data based on the volume data associated with the text data by the conversion unit. A server according to any of the above.

  According to such a configuration, the server associates the volume data indicating the volume of the voice data with the text data, and weights the index of each word included in the text data based on the volume data.

  Accordingly, the server can increase the degree of importance for a phrase with a high volume, so that it is possible to increase the possibility of extracting a phrase emphasized by a speaker as an important word.

(9) A system for extracting important words in the conversation by a terminal device that receives the voice of the conversation and a server capable of data communication via a network,
The server
In response to a processing start request from the terminal device, a specifying unit that specifies the terminal device;
Connection means for establishing a data communication connection with the terminal device specified by the specifying means;
Receiving means for receiving voice data of the conversation;
Separating means for separating voice data received by the receiving means for each speaker;
Conversion means for converting each of the audio data separated by the separation means into text data;
A selection means for selecting a predetermined number of text data from the newest speech time among the text data converted by the conversion means;
Extraction means for extracting an important word based on an index relating to the appearance frequency of each word included in the text data selected by the selection means;
Transmitting means for transmitting the important word extracted by the extracting means to the terminal device in which a data communication connection has been established by the connecting means,
The terminal device
A system comprising display means for displaying the important word transmitted by the transmission means.

  According to such a configuration, the system displays the important word transmitted by the server when the terminal device that receives the voice of the conversation communicates with the server. Thus, by operating the system, the same effects as (1) and (2) can be expected.

  (10) The system according to (9), wherein the terminal device further includes storage means for storing the important words transmitted by the transmission means in time series.

  According to such a configuration, since the terminal device of the system stores the extracted important words in time series, even after the conversation is over, for example, when creating the minutes, etc. You can refer to this conversation flow.

(11) A method for extracting important words in a conversation,
Receiving the voice data of the conversation;
A separation step of separating the voice data received by the reception step for each speaker;
A conversion step of converting each of the audio data separated by the separation step into text data;
A selection step of selecting a predetermined number of text data from the newest speech time among the text data converted by the conversion step;
An extraction step of extracting an important word based on an index relating to an appearance frequency of each word included in the text data selected by the selection step.

  According to such a configuration, the same effect as in (1) can be expected by executing the method.

(12) A program that causes a server to extract important words in a conversation,
Receiving the voice data of the conversation;
A separation step of separating the voice data received by the reception step for each speaker;
A conversion step of converting each of the audio data separated by the separation step into text data;
A selection step of selecting a predetermined number of text data from the newest speech time among the text data converted by the conversion step;
A program for executing an extraction step of extracting an important word based on an index related to an appearance frequency of each word included in the text data selected in the selection step.

  According to such a configuration, the same effect as in (1) can be expected by executing the program.

  According to the present invention, important words in a conversation can be extracted and presented as text data. In addition, advertisements can be distributed effectively to the public.

(First embodiment)
The first embodiment will be described below with reference to the drawings.

[System Overview]
FIG. 1 is a diagram illustrating an overview of a system according to the first embodiment. A terminal device 20 that receives audio in a conversation such as a conference and a server 10 that extracts important words in the conversation are connected via a network. In the first embodiment, the terminal device 20 is described as a mobile phone. However, the terminal device 20 is not limited to this as long as the terminal device has a communication function. Moreover, in order not to limit a use place, it is preferable that it is a portable small terminal device.

  The terminal device 20 transmits voice data of the conversation to the server 10 and receives and displays the important words extracted by the server 10. Thereby, the user of the terminal device 20 can know the important words that change every moment in the conversation in real time.

  The server 10 generates speech text data using speech recognition technology for speech data received from the terminal device 20. Subsequently, the server 10 extracts an important word in the conversation using a TF / IDF technique (details will be described later) in consideration of the freshness of the word for the generated text data.

[Hardware configuration]
FIG. 2 is a diagram illustrating an example of a hardware configuration of the server 10 according to the first embodiment. The server 10 includes a CPU (Central Processing Unit) 1 (1010) constituting the control device 101 (a plurality of CPUs such as CPU 2 (1012) may be added in a multiprocessor configuration), a bus line 1005, a communication I / F 1040, includes a main memory 1050, BIOS (Basic input Output system ) 1060, USB port 1090, I / O controller 107 input device 1100 and a display device 1022 such as a keyboard and mouse to 0 rabbi.

  The BIOS 1060 stores a boot program executed by the control device 101 when the server 10 is started up, a program depending on the hardware of the server 10, and the like.

  A storage device 107 such as a tape drive 1072, a hard disk 1074, an optical disk drive 1076, and a semiconductor memory 1078 can be connected to the I / O controller 1070.

  The hard disk 1074 constituting the storage device 107 stores various programs for the server 10 to function as a server and programs for executing the functions of the present invention, and various databases can be configured as necessary.

  As the optical disk drive 1076, for example, a DVD-ROM drive, a CD-ROM drive, a DVD-RAM drive, a CD-RAM drive, or the like can be used. In this case, the optical disk 1077 corresponding to each drive is used. A program or data can be read from the optical disk 1077 by the optical disk drive 1076 and provided to the main memory 1050 or the hard disk 1074 via the I / O controller 1070. Similarly, the tape medium 1071 corresponding to the tape drive 1072 can be used mainly for backup.

Program provided to the server 10, a hard disk 1074, optical disk 107 7 or is provided stored on a recording medium such as a memory card. The program may be installed in the server 10 and executed by being read from the recording medium via the I / O controller 1070 or downloaded via the communication I / F 1040.

The aforementioned program may be stored in an internal or external storage medium. Here, the storage medium constituting the storage device 107, a hard disk 1074, optical disk 107 7 or to any other memory card, a magneto-optical recording medium such as an MD, a tape medium. Further, a storage device such as a hard disk 1074 or an optical disk library provided in a server system connected to a dedicated communication line or the Internet may be used as a recording medium, and the program may be provided to the server 10 via the communication line.

  Here, the display device 1022 displays a screen for accepting data input to the user, or displays a screen of a calculation processing result by the server 10, and includes a cathode ray tube display device (CRT) and a liquid crystal display device (LCD). ) And the like.

  Here, the input device 1100 accepts input by the user, and may be configured by a keyboard, a mouse, and the like.

  The communication I / F 1040 is a network adapter for enabling the server 10 to be connected to a terminal via a dedicated network or a public network. The communication I / F 1040 may include a modem, a cable modem, and an Ethernet (registered trademark) adapter.

  In the above example, the server 10 has been mainly described. However, the functions described above can also be realized by installing a program in a computer and operating the computer as a server device. Therefore, the functions realized by the server described as an embodiment in the present invention can be realized by executing the above-described method on the computer, or by introducing and executing the above-described program on the computer. It is.

[Function configuration]
FIG. 3 is a diagram illustrating a configuration of main functions of the control device 101 in the server 10 according to the first embodiment. In addition, although each function demonstrated below was implement | achieved in the server 10 single-piece | unit, it is not restricted to this, You may distribute a function to several servers suitably.

  The server 10 includes a telephone number specifying unit 110, a voice receiving unit 120, a person specifying unit 130, a voice recognition unit 140, a weight setting unit 150, a keyword extraction unit 160, and a keyword transmission unit 170. Prepare.

  The telephone number specifying unit 110 specifies the telephone number of the terminal device 20 in response to receiving the processing start request from the terminal device 20. Specifically, the registered user table shown in FIG. 4 is stored in the storage device 107, and the telephone number specifying unit 110 specifies the telephone number of the terminal device 20 corresponding to the user ID, that is, the target for distributing the important word. To do.

  The sound receiving unit 120 receives sound data collected by the terminal device 20 and converted as electronic data.

  The person specifying unit 130 specifies a speaker for the audio data received by the audio receiving unit 120. That is, the voice in the conversation is separated for each speaker, and a plurality of voice data associated with the speaker is obtained in time series. For example, according to the schematic diagram of the text data generated from the conversation shown in FIG. 5, the voice data of “Mr. A”, the voice data of “Mr. B” who spoke next, and the voice of “Mr. C” who spoke next The voice data and the voice data of “Mr. D” who spoke last are separated from each other.

  Here, the person specifying unit 130 can be realized by using an existing technique such as the above-described Patent Document 1.

  The speech recognition unit 140 analyzes the speech data received by the speech reception unit 120 and separated for each speaker by the person identification unit 130, and converts each into text data. Furthermore, the speech recognition unit 140 performs morphological analysis on the converted text data and extracts words. According to the schematic diagram of FIG. 5, for example, “AAA”, “BBX”, and “CCC” are extracted from the utterance of “Mr. A”.

  Here, the speech recognition unit 140 refers to the synonym table shown in FIG. 6 and converts “BBX” into a synonym “BBB”. Thereby, it can be recognized that it is the same as “BBB” extracted from the remarks of “Mr. B”.

  Here, based on existing technology, recognized speech is stored as text data (reference numerals 51, 52, 53, and 54 in FIG. 5) as a combination of words. At this time, it is stored in association with the speaker specified by the person specifying unit 130. Furthermore, sound volume data indicating the sound data volume may be stored in association with each other.

  The weight setting unit 150 selects text data that is a target for extracting important words from the text data converted by the voice recognition unit 140, and further weights each text data with importance.

Specifically, the weight setting unit 150 first selects a predetermined number of text data from the newest utterance time. The predetermined number, be one that is set in advance, for example, if 4 and setting a predetermined number, text data 51,52,5 3 Contact and 54 is selected by the schematic diagram of FIG.

Here, when a topic change word such as “by the way”, “Now”, or “the story is different” appears in the text data, the text data after the topic change word is selected. For example, the predetermined number of even 4, if the topic transformed word appears in the text data 54, text data 51,5 2 Contact and 53 are excluded. As a result, text data related to the currently spoken topic is selected.

  Subsequently, the weight setting unit 150 weights each text data based on the number of speakers, the volume, and the like. For example, the importance is increased for text data of a speaker who has a larger number of utterances than others. Also, the greater the volume data associated with the text data by the voice recognition unit 140, the higher the importance. Or you may make importance high about the thing with relatively large volume data among the text data of the same speaker.

  In addition, the voice recognition unit 140 may associate the volume data with each word / phrase included in the text data. In this case, the weight setting unit 150 can weight the importance of words included in the text data.

Important word extracting unit 160 refers to the text data 51,52,5 3 Contact and 54 selected by the weight setting unit 150, the words contained in each of the indicators of frequency, TF (Term Frequency) and IDF A TF · IDF (t) value that is a product of (Inverse Document Frequency) is calculated.

により算出される。
ＴＦ（ｔ）は、語句ｔがテキストデータに含まれる数を示す。
ＤＦ（ｔ）は、語句ｔが含まれるテキストデータの数を示す。
Ｎは、選択されたテキストデータの数を示す。 The TF · IDF (t) value for the word t is

Is calculated by
TF (t) indicates the number of words / phrases t included in the text data.
DF (t) indicates the number of text data including the word t.
N indicates the number of selected text data.

  For example, according to the schematic diagram of FIG. 5, when the predetermined number “N = 4” is set, “TF (t) = 1” and “DF (t) = 4” are included for “AAA” included in the text data 54. " For “EEE”, “TF (t) = 2” and “DF (t) = 2”.

  That is, the TF · IDF (t) value is larger in “EEE” than “AAA”. The keyword extraction unit 160 extracts the word / phrase having the maximum TF / IDF (t) value as the keyword. It should be noted that not only the word / phrase having the maximum TF · IDF (t) value but also a plurality of words / phrases may be extracted, or words / phrases having a certain value or more may be extracted.

  Here, the keyword extraction unit 160 preferably adjusts the TF (t), DF (t), or TF / IDF (t) value based on the importance weighting set by the weighting setting unit 150. . Thereby, the important word extraction part 160 can extract the important word based on the number of utterances and a sound volume.

  Note that the formula for calculating the TF / IDF (t) value of the first embodiment is an example as an index of importance, and is not limited thereto. For example, the logarithm (log) need not be used, and the base of the logarithm can be set as appropriate, such as “10”, “2”, or the natural logarithm “e”.

  Moreover, although TF (t) is the number of words / phrases t, it is not limited thereto, and may be a value indicating the appearance frequency, such as a ratio included in text data. Similarly, DF (t) may be a value indicating the appearance frequency, such as a ratio of text data including the word t with respect to N.

  The keyword transmission unit 170 transmits the keyword extracted by the keyword extraction unit 160 to the terminal device 20 specified by the telephone number specification unit 110. Since this process may be a one-way communication of character data to the terminal device 20, a so-called push distribution technique can be used. The important word transmission unit 170 establishes a data communication connection with the terminal device 20 specified by the telephone number specifying unit 110 and then extracts the important word from the terminal device 20 each time an important word is extracted by the important word extraction unit 160. Continue delivery.

  Here, the terminal device 20 notifies the user by displaying the important word received from the important word transmission unit 170 of the server 10. Thereby, the user of the terminal device 20 can grasp the important words that change every moment in the conversation in real time.

  Further, the terminal device 20 stores the received important words in time series in association with the time of utterance. Thereby, the user can grasp | ascertain the flow of conversation later.

[Processing flow]
FIG. 7 is a flowchart showing processing of the control device 101 in the server 10 according to the first embodiment.

  In step S <b> 1, the control device 101 receives a process start request from the terminal device 20 together with the user ID.

  In step S2, the control device 101 identifies the telephone number of the terminal device 20 based on the user ID received in step S1.

  In step S <b> 3, the control device 101 receives conversation voice data from the terminal device 20.

  In step S4, the control apparatus 101 specifies the speaker of the voice data received in step S3.

  In step S5, the control apparatus 101 determines whether the speaker specified in step S4 has changed. If this determination is YES, it can be determined that a series of utterances by the same speaker has ended, and the process moves to step S6. On the other hand, if the determination is NO, it can be determined that the utterance by the same speaker is continuing, so the process returns to step S3 and continues to receive the audio data.

  In step S6, the control device 101 converts the voice data received in step S3 into text data.

  In step S7, the control device 101 performs morphological analysis on the text data converted in step S6, and extracts a phrase included in the text data.

  In step S8, the control apparatus 101 weights the word / phrase extracted in step S7. Specifically, as described above, the importance level is adjusted based on the newness of the text data, the number of times the speaker speaks, the volume, and the like.

  In step S9, the control apparatus 101 calculates a TF / IDF (t) value for the word weighted in step S8, and extracts an important word based on this value.

  In step S <b> 10, the control device 101 transmits the important word extracted in step S <b> 9 to the terminal device 20.

  In step S11, the control device 101 determines whether or not to end the process. Specifically, it is determined that the process is ended by receiving the end request from the terminal device 20. If this determination is YES, the process ends. If the determination is NO, the process returns to step S3 and continues to receive audio data.

(Second Embodiment)
Next, a second embodiment will be described. In the second embodiment, video / audio is transmitted from the live relay transmission device to the live advertisement distribution server, and the live advertisement distribution server distributes the video / audio to the terminal device in real time and extracts important words. The advertisement according to the important word is delivered to the terminal device.

  In the following description, the same reference numerals or the same reference numerals are given to the portions that perform the same functions as those in the first embodiment described above, and overlapping descriptions will be omitted as appropriate.

[System Overview]
FIG. 8 is a diagram illustrating an overview of a system according to the second embodiment. Live relay transmission device 30 that transmits live relay video / audio to live advertisement distribution server 200, terminal device 20, and live video / audio to terminal device 20 are distributed in real time, and important words of live audio are extracted. A live advertisement distribution server 200 that distributes advertisements corresponding to the important words to the terminal device 20 in conjunction with real-time distribution is connected via a network. In the second embodiment, the terminal device 20 will be described as having a browser capable of receiving and reproducing live broadcast video / audio distributed in real time.

  The live relay transmission device 30 relays the live and transmits video / audio to the live advertisement distribution server 200.

  When the live advertisement distribution server 200 receives the video / audio from the live relay transmission device 30, the live advertisement distribution server 200 distributes the video / audio to the terminal device 20 in real time and generates voice text data using a voice recognition technology. Subsequently, the live advertisement distribution server 200 extracts an important word in the conversation by using the TF / IDF technology considering the freshness of the word for the generated text data. Then, an advertisement corresponding to the extracted important word is acquired and distributed to the terminal device 20 in conjunction with real-time distribution.

  The terminal device 20 receives and reproduces the video / audio from the live advertisement distribution server 200, and also receives the advertisement and displays it in conjunction with the video / audio.

[Function configuration]
FIG. 9 is a diagram illustrating a configuration of main functions executed by the control device 101 in the live advertisement distribution server 200 according to the second embodiment. In addition, although each function demonstrated below was implement | achieved in the live advertisement delivery server 200 single-piece | unit, it is not restricted to this, You may distribute a function to several servers suitably.

  The live advertisement distribution server 200 includes a video / audio reception unit 121, a person identification unit 130, a voice recognition unit 140, a weighting setting unit 150, a keyword extraction unit 160, an advertisement extraction unit 210, and a distribution advertisement determination unit 220. And an advertisement distribution unit 230 and a real-time distribution unit 240.

  The video / audio receiver 121 receives the video / audio data recorded by the live relay transmitter 30 and converted as electronic data.

  The person specifying unit 130 extracts audio data from the video / audio data received by the audio / video receiving unit 121, and specifies the speaker for the audio data in the same manner as the person specifying unit 130 of the first embodiment. Do.

  The voice recognition unit 140, the weight setting unit 150, and the keyword extraction unit 160 have the same functions as those in the first embodiment.

  The advertisement extraction unit 210 extracts an advertisement corresponding to the keyword extracted by the keyword extraction unit 160 with reference to the advertisement table shown in FIG. Specifically, matching is performed between an important word and a keyword stored in the advertisement table. Then, in the matching, an advertisement including a keyword that matches the keyword is extracted.

  FIG. 10 is a diagram illustrating an advertisement table according to the second embodiment. The advertisement table stores an advertisement ID, a keyword, advertisement contents, a URL, and the like. The advertisement ID is a key for specifying the stored advertisement. The keyword is used for matching with the keyword extracted by the keyword extraction unit 160. The advertisement content and URL are information distributed as an advertisement to the terminal device 20 and are displayed together with a live video.

  Returning to FIG. 9, when the number of advertisements extracted by the advertisement extraction unit 210 is less than the predetermined number of display advertisements, the distribution advertisement determination unit 220 determines and extracts the extracted advertisements as distribution advertisements. If the number of ads that are displayed is greater than or equal to the number of displayed ads, the number of ads to be distributed is selected at random from the number of extracted ads, or the ads with the highest bid amount are selected, Adjustments are made based on predetermined rules. Note that the number of displayed advertisements is the number of advertisements distributed together with live video / audio real-time distribution, and can be determined as appropriate.

  The advertisement distribution unit 230 distributes the advertisement determined to be distributed by the distribution advertisement determination unit 220 to the terminal device 20.

  The real-time distribution unit 240 distributes the video / audio data received by the video / audio reception unit 121 to the terminal device 20 in real time.

[Flow chart of advertisement distribution processing]
FIG. 11 is a flowchart showing an advertisement distribution process executed by the control device 101 in the live advertisement distribution server 200 according to the second embodiment. In the live advertisement distribution server 200, the video / audio real-time distribution process is performed by the control device 101 separately from the advertisement distribution process.

  In step S <b> 101, the control apparatus 101 receives conversation video / audio data from the live relay transmission apparatus 30.

  In step S102, the control apparatus 101 extracts audio data from the video / audio data received in step S101, and identifies a speaker for the audio data.

  In step S103, the control apparatus 101 determines whether the speaker specified in step S102 has changed. If this determination is YES, it can be determined that a series of utterances by the same speaker has ended, and the process moves to step S104. On the other hand, if the determination is NO, it can be determined that the utterance by the same speaker is continuing, so the process returns to step S101 to continue receiving the audio data.

  In step S104, the control device 101 converts the voice data received in step S101 into text data.

  In step S105, the control device 101 performs morphological analysis on the text data converted in step S104, and extracts a phrase included in the text data.

  In step S106, the control apparatus 101 weights the words extracted in step S105. Specifically, as described above, the importance level is adjusted based on the newness of the text data, the number of times the speaker speaks, the volume, and the like.

  In step S107, the control device 101 calculates a TF / IDF (t) value for the word weighted in step S106, and extracts an important word based on this value.

  In step S108, the control device 101 extracts an advertisement corresponding to the important word extracted in step S107 from the storage device 107.

In step S109, the control device 101 distributes the advertisement extracted in step S108 to the terminal device 20 .

  In step S110, the control device 101 determines whether or not to end the process. Specifically, it is determined that the process is to be terminated when a termination request is received from the live relay transmitter 30. If this determination is YES, the process ends. If the determination is NO, the process returns to step S101 to continue receiving video / audio data.

[Display example of advertisement delivery]
12 and 13 are diagrams illustrating display examples of advertisement distribution in the terminal device 20 according to the second embodiment. In the description of FIG. 12 and FIG. 13, the number of display advertisements in the distribution advertisement determination unit 220 is “3”.

  FIG. 12 is a diagram illustrating a state of live relay. In FIG. 12, the state where Mr. E and Mr. F have a conversation is being relayed live, and the live relay transmission device 30 transmits the live relay video / audio data to the live advertisement distribution server 200. Here, the contents of the conversation between Mr. E and Mr. F are indicated by balloons 55 to 58, balloons 55 and 57 are comments of Mr. E, and balloons 56 and 58 are comments of Mr. F.

  The live advertisement distribution server 200 receives live broadcast video / audio data by the video / audio reception unit 121. Then, the voice data is extracted by the person specifying unit 130, and the speaker is specified for the voice data. In FIG. 12, the voice data of “Mr. E” and “Mr. F” are separated. Then, the speech recognition unit 140 analyzes the speech data separated for each speaker and converts each into text data. Then, morphological analysis is performed on the converted text data to extract words. According to balloons 55 to 58 in FIG. 12, “hobby”, “drive”, and “holiday” are extracted. Then, the speech recognition unit 140 converts the text data for the synonym, and the weighting setting unit 150 weights the text data. Here, four text data converted from the speech shown in speech bubbles 55-58 are selected.

  The keyword extraction unit 160 refers to the four text data selected by the weight setting unit 150 and calculates a TF / IDF (t) value. Here, as in the example of the keyword extraction unit 160 of the first embodiment, when the predetermined number “N = 4”, the phrase “drive” included in the text data corresponding to the speech balloon 58 is “TF ( t) = 3 ”and“ DF (t) = 3 ”, and the TF / IDF (t) value is extracted as the word having the maximum value, that is, the important word.

  The advertisement extraction unit 210 refers to the advertisement table (FIG. 10) and extracts an advertisement in which “drive” extracted as an important word is included in the keyword. Here, advertisements with advertisement IDs “101” and “102” are extracted. Since the number of extracted advertisements is “2” and smaller than the number of displayed advertisements, that is, “3”, the distribution advertisement determination unit 220 is determined as an advertisement to which advertisements with advertisement IDs “101” and “102” are distributed. The Then, the advertisements with the advertisement IDs “101” and “102” are distributed to the terminal device 20 by the advertisement distribution unit 230.

  FIG. 13 is a diagram illustrating a display example when an advertisement corresponding to an important word is distributed to the terminal device 20 according to the second embodiment.

  In FIG. 13, the browser 301 is displayed on the display unit provided in the terminal device 20. Then, it can be confirmed that the live video 302 is displayed on the browser 301. Further, it can be confirmed that the advertisement 303 and the advertisement 304 are displayed as sponsor advertisements on the right side of the live video 302. The advertisements 303 and 304 are advertisements distributed from the live advertisement distribution server 200 by the advertisement distribution unit 230, and the advertisement contents and URLs related to the advertisement IDs “101” and “102” in the advertisement table (FIG. 10) are the same. You can see each displayed.

  In this way, since an important word in a live relay conversation is extracted and an advertisement corresponding to the important word is distributed together with the live relay, the advertisement is harmonized with the live relay, and an advertisement display without a sense of incongruity can be realized. When the user of the terminal device 20 is a fan of a live broadcast performer, it is possible to expect a situation where the probability of clicking on the advertisement display is high, that is, a high conversion rate. Furthermore, since the displayed advertisement is an advertisement in harmony with the live broadcast video / audio, it tends to remain in the user's impression, and a high advertising effect can be expected.

  As mentioned above, although embodiment of this invention was described, this invention is not restricted to embodiment mentioned above. The effects described in the embodiments of the present invention are only the most preferable effects resulting from the present invention, and the effects of the present invention are limited to those described in the embodiments of the present invention. is not.

It is a figure which shows the outline | summary of the system which concerns on 1st Embodiment. It is a figure which shows an example of the hardware constitutions of the server 10 which concerns on 1st Embodiment. It is a figure which shows the structure of the main functions of the control apparatus 101 in the server 10 which concerns on 1st Embodiment. It is a figure which shows the registered user table which concerns on 1st Embodiment. It is a schematic diagram of the text data produced | generated from the conversation which concerns on 1st Embodiment. It is a figure which shows the synonym table which concerns on 1st Embodiment. It is a flowchart which shows the process of the control apparatus 101 in the server 10 which concerns on 1st Embodiment. It is a figure which shows the outline | summary of the system which concerns on 2nd Embodiment. It is a figure which shows the structure of the main functions performed by the control apparatus 101 in the live advertisement delivery server 200 which concerns on 2nd Embodiment. It is a figure which shows the advertisement table which concerns on 2nd Embodiment. It is a flowchart which shows the advertisement delivery process performed by the control apparatus 101 in the live advertisement delivery server 200 which concerns on 2nd Embodiment. It is a figure which shows the mode of the live relay which concerns on 2nd Embodiment. It is a figure which shows the example of a display when the advertisement according to an important word is delivered with respect to the terminal device 20 which concerns on 2nd Embodiment.

DESCRIPTION OF SYMBOLS 10 Server 20 Terminal apparatus 30 Live relay transmission apparatus 101 Control apparatus 107 Storage apparatus 110 Telephone number specific | specification part 120 Audio | voice receiving part 121 Image | video audio | voice receiving part 130 Person specific part 140 Voice recognition part 150 Weight setting part 160 Important word extraction part 170 Important word Transmission unit 200 Live advertisement distribution server 210 Advertisement extraction unit 220 Distribution advertisement determination unit 230 Advertisement distribution unit 240 Real-time distribution unit

Claims (11)

A server that extracts important words in a conversation,
Receiving means for receiving voice data of the conversation;
Separating means for separating voice data received by the receiving means for each speaker;
Conversion means for converting each of the audio data separated by the separation means into text data;
A selection means for selecting a predetermined number of text data from the newest speech time among the text data converted by the conversion means;
Extraction means for extracting an important word based on an index relating to the appearance frequency of each word included in the text data selected by the selection means,
The converting means associates volume data indicating the volume of the audio data with the text data,
The extracting means includes a TF value indicating the number of each word appeared in the text data before Symbol predetermined number, the product of the IDF value is the reciprocal of DF value indicating the number of text data containing each word Is calculated as the index, and the index is adjusted by weighting according to importance according to volume data associated with the text data . The server according to claim 1, wherein the extraction unit increases the importance of text data of the same speaker whose volume data is relatively large. In response to a processing start request from a terminal device connected via a network, a specifying unit that specifies the terminal device;
Connection means for establishing a data communication connection with the terminal device specified by the specifying means;
The server according to claim 1, further comprising: a transmission unit that transmits the important word extracted by the extraction unit to the terminal device in which a data communication connection has been established by the connection unit. Real-time distribution means for distributing video and audio of the conversation in real time to a terminal device connected via a network;
Advertisement acquisition means for acquiring advertisement information in accordance with the important words extracted by the extraction means;
The server according to claim 1, further comprising: an advertisement distribution unit that distributes the advertisement information acquired by the advertisement acquisition unit to the terminal device in conjunction with the real-time distribution. The selection means, when detecting that the text data includes a predetermined type of word, selects the text data after the predetermined type of word is spoken. Item 5. The server according to any one of Items 4 . A synonym database for determining a synonym of each word included in the text data;
The server according to any one of claims 1 to 5 , wherein the extraction unit calculates the index related to the appearance frequency of each word including the synonyms stored in the synonym database. The extraction means weights the index of each word included in the text data of the speaker based on the number of text data with the same speaker in the text data selected by the selection means. server according to any one of claims 1 to 6, which comprises carrying out. A system for extracting important words in the conversation by a terminal device that receives the voice of the conversation and a server capable of data communication via a network,
The server
In response to a processing start request from the terminal device, a specifying unit that specifies the terminal device;
Connection means for establishing a data communication connection with the terminal device specified by the specifying means;
Receiving means for receiving voice data of the conversation;
Separating means for separating voice data received by the receiving means for each speaker;
Conversion means for converting each of the audio data separated by the separation means into text data;
A selection means for selecting a predetermined number of text data from the newest speech time among the text data converted by the conversion means;
Extraction means for extracting an important word based on an index relating to the appearance frequency of each word included in the text data selected by the selection means;
Transmitting means for transmitting the important word extracted by the extracting means to the terminal device in which a data communication connection has been established by the connecting means,
The terminal device
Display means for displaying the important word transmitted by the transmission means;
The converting means associates volume data indicating the volume of the audio data with the text data,
The extracting means includes a TF value indicating the number of each word appeared in the text data before Symbol predetermined number, the product of the IDF value is the reciprocal of DF value indicating the number of text data containing each word Is calculated as the index, and the index is adjusted by weighting according to importance according to volume data associated with the text data .   9. The system according to claim 8, wherein the terminal device further comprises storage means for storing the important words transmitted by the transmission means in time series. A computer that extracts important words in a conversation,
Receiving the voice data of the conversation;
A separation step of separating the voice data received by the reception step for each speaker;
A conversion step of converting each of the audio data separated by the separation step into text data;
A selection step of selecting a predetermined number of text data from the newest speech time among the text data converted by the conversion step;
Extracting an important word based on an index relating to the appearance frequency of each word included in the text data selected by the selection step, and
In the conversion step, volume data indicating the volume of the audio data is associated with the text data,
In the extraction step, the TF value indicating the number of each word appeared in the text data before Symbol predetermined number, the product of the IDF value is the reciprocal of DF value indicating the number of text data containing each word Is calculated as the index, and the index is adjusted by weighting according to importance according to volume data associated with the text data . A program that causes a computer to extract important words in a conversation,
Receiving the voice data of the conversation;
A separation step of separating the voice data received by the reception step for each speaker;
A conversion step of converting each of the audio data separated by the separation step into text data;
A selection step of selecting a predetermined number of text data from the newest speech time among the text data converted by the conversion step;
An extraction step of extracting an important word based on an index relating to the appearance frequency of each word included in the text data selected by the selection step;
In the conversion step, volume data indicating the volume of the audio data is associated with the text data,
In the extraction step, the TF value indicating the number of each word appeared in the text data before Symbol predetermined number, the product of the IDF value is the reciprocal of DF value indicating the number of text data containing each word Is calculated as the index, and the index is adjusted by weighting according to importance according to volume data associated with the text data .

Images