JP2021189348A - Voice dialogue device, voice dialogue method, and voice dialogue program - Google Patents

Abstract

To provide a voice dialogue device, a voice dialogue method, and a voice dialogue program for performing a voice dialogue which smooths a dialogue with a user.SOLUTION: A voice dialogue device 300 includes a processor which executes a program, and a storage device which stores the program. A functional configuration by the processor includes a reliability acquisition unit which acquires reliability of each word which constitutes a word string related to an uttered voice, and a repeat questioning sentence generator to generate a repeat questioning sentence to hear back to an utterance source of an uttered voice, based on reliability acquired by the acquisition process.SELECTED DRAWING: Figure 4

Description

The present invention relates to a voice dialogue device, a voice dialogue method, and a voice dialogue program for performing a voice dialogue.

Communication robots capable of voice dialogue with humans often accept voice input including errors, as compared with text input dialogue systems. There are various patterns of voice input including this error, for example, voice section detection error, voice recognition error, recognition of irrelevant voice, stagnation, and broken utterance due to correction utterance in the middle of utterance. Therefore, when such a voice input is accepted, a dialogue with a human cannot be established unless the communication robot determines whether or not the input voice is normal and performs an appropriate exception handling process.

Patent Document 1 and Patent Document 2 are such voice dialogue techniques. Patent Document 1 discloses a robot that notifies a subject by voice of voice data corresponding to a phrase whose voice recognition accuracy is less than a predetermined value. This robot acquires the utterance voice from the target person, and among the voice data of the acquired utterance voice, the voice data corresponding to the phrase whose voice recognition accuracy does not reach the predetermined value is voiced to the target person. Output.

Patent Document 2 discloses a dialogue control device that controls the dialogue device so as to perform a confirmation operation performed by a human and reduces an erroneous response of the dialogue device. This dialogue control device outputs a voice that triggers a dialogue from the dialogue device side to start a dialogue, a response scenario that responds to a speech from the user side, and a dialogue with the user. A scenario storage unit that stores a confirmation scenario that confirms whether or not to start, a talk start index S that outputs a voice that triggers the dialogue from the dialogue device side, and indicates whether or not the dialogue should be started. The response start index R, which indicates whether or not a response should be made to a certain voice, is used as an input, J and K are set to any of integers of 1 or more, and the magnitude of the talk start index S and the thresholds of J is set. It includes a scenario selection unit that selects a talking scenario, a response scenario, or a confirmation scenario based on the relationship and the magnitude relationship between the response start index R and K thresholds.

Japanese Unexamined Patent Publication No. 2018-81147 Japanese Unexamined Patent Publication No. 2018-87847

However, in the above-mentioned Patent Documents 1 and 2, it is not considered which part of the input dialogue voice can be recognized and which part cannot be recognized by the communication robot. Therefore, the user who utters the dialogue voice cannot know which part is transmitted to the communication robot and which part is not transmitted. Therefore, the user may be asked the same question over and over again, which may increase the annoyance of the user and disrupt the dialogue with the user.

An object of the present invention is to facilitate dialogue with a user.

The voice dialogue device according to one aspect of the invention disclosed in the present application is a voice dialogue device having a processor for executing a program and a storage device for storing the program, and the processor is a word string related to spoken voice. Based on the reliability of each word constituting the It is a feature.

According to a typical embodiment of the present invention, it is possible to facilitate dialogue with the user. Issues, configurations and effects other than those described above will be clarified by the description of the following examples.

FIG. 1 is an explanatory diagram showing an example of voice dialogue between a user and a communication robot according to the first embodiment. FIG. 2 is an explanatory diagram showing a system configuration example of the voice dialogue system according to the first embodiment. FIG. 3 is a block diagram showing a hardware configuration example of the voice dialogue device. FIG. 4 is a block diagram showing a functional configuration example of the voice dialogue device according to the first embodiment. FIG. 5 is an explanatory diagram showing an example of the voice recognition result. FIG. 6 is an explanatory diagram showing an example of conversion from a word string to a part-of-speech string. FIG. 7 is an explanatory diagram 1 showing an example of a part-speech classification definition of a part-speech sequence. FIG. 8 is an explanatory diagram 2 showing an example of a part-speech classification definition of a part-speech sequence. FIG. 9 is an explanatory diagram showing an example of reliability calculation by the reliability acquisition unit. FIG. 10 is a flowchart showing an example of a voice dialogue processing procedure by the voice dialogue device according to the first embodiment. FIG. 11 is a flowchart showing a detailed processing procedure example of the hearing back sentence generation processing (step S1005) shown in FIG. FIG. 12 is a flowchart showing a detailed processing procedure example of the mask word estimation process (step S1106) shown in FIG. FIG. 13 is an explanatory diagram showing an example of the mask word estimation process (step S1106) shown in FIG. FIG. 14 is a flowchart showing a detailed processing procedure example of the rephrasing utterance interpretation processing (step S1007) shown in FIG. FIG. 15 is a flowchart showing a detailed processing procedure example of the dialogue control processing (steps S1008 and S1403) shown in FIGS. 10 and 14. FIG. 16 is a flowchart showing an example of the flow of dialogue between the user and the voice dialogue device. FIG. 17 is a block diagram showing a functional configuration example of the voice dialogue device according to the second embodiment. FIG. 18 is a flowchart showing an example of a voice dialogue processing procedure by the voice dialogue device according to the second embodiment.

<Example of voice dialogue>
FIG. 1 is an explanatory diagram showing an example of voice dialogue between a user and a communication robot (hereinafter, simply “robot”) according to the first embodiment. FIG. 1 shows the hearing patterns P1 to P5 from the robot 102 when the user 101 speaks to the robot 102 with the utterance voice 110 "What is the sunrise time of Mt. Fuji tomorrow?". It should be noted that the size of the uttered voice 110, the utterance speed, the accuracy of pronunciation, and at least one of the surrounding environments are different for each of the following (A) to (E).

(A) shows an example of a response in the listening pattern P1. Specifically, for example, the robot 102 voice-recognizes the utterance voice 110 as "Tomorrow's sunrise of Mt. Fuji 〇 × △ □" ("○ × △ □" is a part that cannot be recognized due to low reliability). , Response sentence to hear back the high part (of the sunrise of Mt. Fuji tomorrow) whose reliability is above the threshold (hereinafter, the answer sentence) "I couldn't hear it well. What is" Sunrise of Mt. Fuji tomorrow "? Is generated as the echo pattern P1 and is spoken to the user 101 who is the source of the speech.

(B) shows a response example in the listening back pattern P2. Specifically, for example, the robot 102 sets the utterance voice 110 to "○ × △ Mt. Fuji 〇 × △ □ time?" ("○ × △" and "○ × △ □" have thresholds of reliability, respectively. Recognize the words "Mt. Fuji" and "Time" with high reliability above the threshold by utterance recognition as "less than the part that could not be recognized low") "Mt. Fuji"? "Time"? Is generated as a back-listening pattern P2, and is spoken to the user 101.

(C) shows a response example in the listening back pattern P3. Specifically, for example, the robot 102 cannot recognize the utterance voice 110 as "What is the sunrise time of tomorrow's 〇 × △ □?" ("○ × △ □" has a reliability lower than the threshold value and cannot be recognized. The voice is recognized as the part), and the estimation result (Mt. Fuji) of the part (○ × △ □) whose reliability is lower than the threshold value and the utterance voice 110 uttered by the user 101 including the estimation result are confirmed. The sentence "Did you say" What is the sunrise time of Mt. Fuji tomorrow? "" Is generated as a listening pattern P3 and spoken to user 101.

(D) shows an example of a response in the listening pattern P4. Specifically, for example, the robot 102 recognizes the uttered voice 110 with high accuracy, and asks back to confirm the entire recognized question. "Did you say" What is the sunrise time of Mt. Fuji tomorrow? " Is generated as a back-listening pattern P4 and is spoken to the user 101.

(E) shows an example of a response in the listening pattern P5. Specifically, for example, the robot 102 cannot recognize the spoken voice 110, and generates a reply sentence "I could not hear it well, please try again." As a reply pattern P5, and speaks to the user 101. do.

The robot 102 calculates the language likelihood (Perplexity of the language model) in addition to the reliability of the voice recognition for the voice recognition result of the spoken voice 110, and controls the normal dialogue by the reliability of the voice recognition and the language likelihood. The response in the above and the generation of the speech-back sentence by the speech-back patterns P1 to P5 are selected.

In this way, the robot 102 informs the user 101 that part or all of the spoken voice 110 could not be recognized, prompts the user 101 for the necessary recurrence, and smoothly advances the dialogue with the user 101. be able to. Further, the user 101 can know which part of the utterance voice 110 is transmitted and which part is not transmitted.

<Voice dialogue system>
FIG. 2 is an explanatory diagram showing a system configuration example of the voice dialogue system according to the first embodiment. The voice dialogue system 200 is, for example, a client-server system, and includes a server 201 and an information processing device 104 such as a robot 102 and a smartphone 103. The server 201 and the information processing device 104 can communicate with each other via a network 202 such as the Internet, a LAN (Local Area Network), and a WAN (Wide Area Network).

In the case of a client-server system, the voice dialogue program is installed on the server 201. Therefore, the server 201 executes voice recognition processing, voice recognition reliability calculation, language likelihood calculation, and response sentence generation as a voice dialogue device. In this case, the information processing apparatus 104 inputs the utterance voice 110, converts the input utterance voice 110 into data, transmits the voice data by the conversion to the server 201, receives the response text from the server 201, and utters the response text. It becomes an interactive interface to execute.

On the other hand, in the stand-alone type, the voice dialogue program is installed in the information processing apparatus 104, and the server 201 is unnecessary. Therefore, the information processing apparatus 104, as a voice dialogue device, inputs the utterance voice 110, converts the input utterance voice 110 data, performs voice recognition processing, calculates the reliability of voice recognition, calculates the language likelihood, generates a response sentence, and responds. Perform the utterance of a sentence.

<Hardware configuration example of voice dialogue device>
FIG. 3 is a block diagram showing a hardware configuration example of the voice dialogue device. The voice dialogue device 300 includes a processor 301, a storage device 302, an input device 303, an output device 304, and a communication interface (communication IF) 305. The processor 301, the storage device 302, the input device 303, the output device 304, and the communication IF 305 are connected by the bus 306. The processor 301 controls the voice dialogue device 300. The storage device 302 serves as a work area for the processor 301. Further, the storage device 302 is a non-temporary or temporary recording medium for storing various programs and data. Examples of the storage device 302 include a ROM (Read Only Memory), a RAM (Random Access Memory), an HDD (Hard Disk Drive), and a flash memory. The input device 303 inputs data. The input device 303 includes, for example, a keyboard, a mouse, a touch panel, a numeric keypad, a scanner, a microphone, and a biosensor. The output device 304 outputs data. The output device 304 includes, for example, a display, a printer, and a speaker. The communication IF 305 connects to the network 202 and transmits / receives data.

<Example of functional configuration of voice dialogue device 300>
FIG. 4 is a block diagram showing a functional configuration example of the voice dialogue device 300 according to the first embodiment. The voice dialogue device 300 is a computer in which a voice dialogue program is installed. The speech dialogue device 300 has access to a speech recognition model 411, a standard language model 412, a field-specific language model 413, a question sentence language model 414, a dialogue context language model 415, and a dialogue knowledge DB 416 (database) 416. These are stored in the storage device 302 of the voice dialogue device 300 shown in FIG. 3 or the storage device 302 of another computer capable of communicating with the voice dialogue device 300.

The speech recognition model 411 is a statistical model of the correspondence between the acoustic features and the phonemes, the correspondence between the phoneme sequences and the words, and the word strings. As the speech recognition model 411, for example, there is a weighted finite state transducer (WFST) that combines an acoustic model and a language model into one object. As an acoustic model, for example, there is a hybrid acoustic model (DNN-HMM) of a Deep Natural Network (DNN) and a hidden Markov model (HMM). In addition, there is N-gram in the language model.

The standard language model 412 is a statistical model of a sequence of words obtained from texts in various fields including dialogue sentences. The standard language model 412 may be, for example, the word N-gram or a recurrent neural network (RNN). The standard language model 412 is used to evaluate the plausibility of a particular language (Japanese in this example).

The field-specific language model 413 is a statistical model of a sequence of words obtained from a text about the operating environment of the robot 102 (for example, if the operating environment is a tourist guide, a text about the guide to the tourist spot) or a collection of assumed questions. .. The field-specific language model 413 may be, for example, the word N-gram or RNN. The field-specific language model 413 is used to evaluate the plausibility of the content spoken in the operating environment. The standard language model 412 may be a language model in which field-specific language models 413 of a plurality of fields are aggregated, or may be a language model related to general spoken language excluding the field-specific language model 413.

The question sentence language model 414 is a statistical model of a sequence of words at the part of speech level obtained from a collection of assumed questions and examples of questions actually received from the user 101 so far. The question sentence language model 414 may be, for example, the part of speech N-gram or RNN. The question sentence language model 414 is a language model used for determining whether or not the utterance of the user 101 is a question from the sequence of words.

The dialogue context language model 415 is a statistical model in which the statistics of arbitrary words and the word groups appearing around them and the statistics of the sequence of phrases are learned for texts in all fields including dialogue sentences. As a dialogue context language model 415, there is a Bidirectional Encoder Representations from Transformers (BERT).

The dialogue knowledge DB 416 is a database that stores examples of assumed question sentences and past question sentences in association with their answers.

Further, the voice dialogue device 300 includes a voice recognition unit 401, a language likelihood acquisition unit 402, a reliability acquisition unit 403, a listening back determination unit 404, a dialogue control unit 405, a listening back sentence generation unit 406, and a dialogue history. It has a management unit 407 and a voice synthesis unit 408. Specifically, these are realized, for example, by causing the processor 301 to execute a voice dialogue program stored in the storage device 302 shown in FIG.

The voice recognition unit 401 inputs the waveform data of the spoken voice 110, and converts the waveform data into the voice feature amount X by using the voice recognition model 411. Specifically, for example, the voice recognition unit 401 cuts out the waveform data with a length of about 25 [msec] and converts it into a voice feature amount vector x. For the voice feature amount vector x, for example, a 13-dimensional mel frequency cepstrum coefficient (MFCC) is used.

The voice recognition unit 401 converts the entire section of the waveform data while shifting the voice feature amount vector x by about 10 [msec], and generates the voice feature amount X = [x_1, ..., x_t, ..., X_T]. x_t is a voice feature amount vector converted in the t-th interval of the waveform data. T is the number of intervals of the waveform data. Then, the voice recognition unit 401 obtains the most probable word string Wt under the condition that the voice feature amount X is given by the following equation (1). W is a word string.

The voice recognition unit 401 outputs the word string Wt to the language likelihood acquisition unit 402. Further, the voice recognition unit 401 generates a word lattice and outputs it to the reliability acquisition unit 403. The word string Wt and the word lattice are used as speech recognition results.

FIG. 5 is an explanatory diagram showing an example of the voice recognition result. The speech recognition result 500 includes the word string Wt and the word lattice WL. The word string Wt (as well as the word string W) is composed of the notation, reading, and part of speech of each of one or more words. When the word string Wt is "today", "ha" and "sunny", the word string Wt is the notation "today", the reading "kyo", and the part of speech "noun" and the word "ha" of the word "today". It is composed of the notation "ha", the reading "wa", and the part of speech "auxiliary", and the notation "sunny", the reading "halle", and the part of speech "noun" of the word "sunny".

The speech recognition model 411 and each language model (excluding the dialogue context language model 415) have the same specifications for word breaks and part-of-speech systems. The BERT used in the dialogue context language model 415 automatically determines word breaks when learning the dialogue context language model 415.

The word lattice WL is network structure data of hypothetical candidates for speech recognition. A numerical value that is the sum of the acoustic likelihood and the language likelihood is given for each word. For example, the sum of the acoustic and linguistic likelihoods for word A (today) is "0.7", and the sum of the acoustic and linguistic likelihoods for word B (ha) is. It is "0.6", and the total value of the acoustic likelihood and the language likelihood for the word C (sunny) is "0.7". The sequence of words in the route that maximizes the sum of these numerical values is the word string Wt.

The voice recognition unit 401 may transfer the waveform data of the user 101 dialogue voice to another computer capable of communicating with the voice dialogue device 300, and acquire the voice recognition result 500 generated by the other computer.

Returning to FIG. 4, the language likelihood acquisition unit 402 uses a plurality of language models for the word string Wt input from the speech recognition unit 401, and the language likelihood (test set perplexi) in each language model. T) is calculated. The plurality of language models are a standard language model 412, a field-specific language model 413, a question sentence language model 414, and a dialogue context language model 415. Of these, the standard language model 412 and the question sentence language model 414 are essential language models for the language likelihood acquisition unit 402.

The probability of occurrence of the word string W1n when N-gram (word N-gram, part word N-gram) is used is, for example, as shown in the following equation (2), and the occurrence probability of the word string W1n when RNN is used. The probability is as shown in the following formula (3), and the probability of occurrence of the word string W1n when BERT is used is as shown in the following formula (4). The word string W1n means an array of words from the word W1 to the word Wn (n is the total number of words included in the word string W1n), and here, the word string W1n = the word string Wt. The language likelihood acquisition unit 402 outputs the language likelihood calculated for each language model to the listening back determination unit 404.

When the language likelihood acquisition unit 402 calculates the language likelihood using the question sentence language model 414 of the part of speech N-gram by the above equation (2), the word sequence Wt is converted into a part of speech sequence. Since each word in the word string Wt is composed of notation, reading, and part of speech, the language likelihood acquisition unit 402 deletes the notation and reading from each word in the word string Wt, and the word string consisting only of part of speech. That is, it is converted into a part of speech string.

FIG. 6 is an explanatory diagram showing an example of conversion from a word string to a part-of-speech string. In the case of the word string Wt, it is converted into a part of speech string of noun → particle → noun. The language likelihood acquisition unit 402 considers the part-speech sequence as the word sequence W1n and applies the above equation (2) to calculate the language likelihood. The language likelihood acquisition unit 402 assigns inflected forms to part of speech that have inflected forms and basic forms (words) to pronouns and particles so that information is not lost more than necessary due to conversion to part of speech strings. May be good.

7 and 8 are explanatory diagrams showing an example of the part-speech classification definition 700 of the part-speech sequence, and follow the definition of the short unit (part-speech classification of the morphological analysis dictionary) defined by the National Institute for Japanese Language and Linguistics. The part-of-speech classification definition 700 is, for example, information stored in the storage device 302. The part-speech classification definition 700 has a major classification 701, a middle classification 702, a minor classification 703, and supplementary information 704. The major classification 701 defines the part of speech contained in the words W1 to Wn. The middle classification 702 defines the subdivided items of the major classification 701. The sub-classification 703 defines the subdivided items of the middle classification 702. Supplementary information 704 defines information to be supplemented to the major classification 701 to the minor classification 703. Part of speech of each word W1 to Wn in the word string Wt contains information such as major classification 701 to minor classification 703, part of speech with inflected forms is inflected, and Katakana words derived from English words are written in English. And.

When converting the word string Wt to the part of speech string, the language likelihood acquisition unit 402 refers to the part of speech classification definition 700, leaves only the information in the part of speech classification definition 700 for each part of speech, and describes the basic form in the supplementary information 704. A basic form (word) is given to a part of speech with. As a result, the lack of information in the part of speech sequence is suppressed.

The language likelihood acquisition unit 402 transfers the word string Wt input from the voice recognition unit 401 to another computer capable of communicating with the voice dialogue device 300, and the other computer calculates based on the word string Wt. You may get the language likelihood for each language model.

Returning to FIG. 4, the reliability acquisition unit 403 determines the speech recognition reliability (hereinafter, simply “reliability”) of each word W1 to Wn constituting the word string Wt and the word string Wt based on the word lattice WL. calculate. Specifically, for example, the reliability is an index value indicating how reliable the word string Wt obtained by the voice recognition unit 401 is. Here, it is assumed that the higher the value, the higher the reliability.

FIG. 9 is an explanatory diagram showing an example of reliability calculation by the reliability acquisition unit 403. The reliability acquisition unit 403 first converts the word lattice WL into the network 600. For a word having only one state transition (for example, word E), the reliability acquisition unit 403 inserts a state transition having an empty symbol ε having a word reliability of 1.0, and inserts the state transition into word E. Align with the state transition of.

Next, the reliability acquisition unit 403 normalizes the reliability of each word A to F in the network 600 by 1.0 for each section and converts it into the word confusion network 601. Further, the reliability acquisition unit 403 calculates the harmonic mean of the reliabilitys of the normalized words A to F in the word confusion network 601 and uses it as the reliability of the word string Wt. The reliability acquisition unit 403 outputs the reliability of the normalized words A to F and the reliability of the word string Wt to the listening back determination unit 404.

The reliability acquisition unit 403 transfers the word lattice WL input from the voice recognition unit 401 to another computer capable of communicating with the voice dialogue device 300, and the other computer calculates based on the word lattice WL. You may get the reliability.

Returning to FIG. 4, the listening back determination unit 404 determines whether or not the listening back to the user 101 should be performed. Specifically, for example, the listening back determination unit 404 uses the language likelihood for each language model from the language likelihood acquisition unit 402 and the reliability of the word string Wt from the reliability acquisition unit 403 to control the dialogue. It is determined which of the dialogue control by 405 and the hearing back processing by the hearing back sentence generation unit 406 is to be executed.

A threshold value is set for each of the language likelihood for each language model, the reliability of the word string Wt, and the reliability of each normalized word W1 to Wn. If any one of the language likelihood and the reliability of the word string Wt for each language model is less than the threshold value, the listening back determination unit 404 executes the listening back sentence generation process. The reliability of each of the normalized words W1 to Wn is used by the recurrence sentence generation unit 406.

When the dialogue control unit 405 determines that it is not necessary to listen back to the user 101 by the listening back determination unit 404, the dialogue control unit 405 refers to the dialogue knowledge DB 416 and generates a response sentence corresponding to the voice recognition result 500 of the spoken voice 110.

The back-to-back sentence generation unit 406 generates a back-to-back sentence for the spoken voice 110 of the user 101 when it is determined by the back-to-back determination unit 404 that the user 101 should be heard back. Specifically, for example, the listening back sentence generation unit 406 generates a listening back sentence according to the determination result of the listening back determination unit 404 (which language model has a language likelihood of less than the threshold value). For example, if the confidence of only some words in the word string Wt is less than the threshold, the dialogue context language model 415 is used to estimate the plausible words that apply to that part, and the user 101 estimates the estimated words. Generates a response sentence as a response sentence that asks whether or not the person has spoken.

The dialogue history management unit 407 stores the word string Wt of the spoken voice 110 of the user 101 and the response sentence generated by the voice dialogue device 300. The voice synthesis unit 408 converts the response sentence generated by the dialogue control unit 405 or the listening back sentence generation unit 406 into voice and outputs it. When the voice dialogue device 300 is realized by the server 201 of the client server system, the voice dialogue device 300 does not have the voice synthesis unit 408, and the response sentence generated by the dialogue control unit 405 or the listening back sentence generation unit 406 is generated. , Is transmitted to an information processing device 104 such as a robot 102 or a smartphone 103 that serves as a client. In this case, the client has a voice synthesis unit 408, and converts the response sentence received from the server 201 into voice and outputs it.

<Example of voice dialogue processing procedure>
FIG. 10 is a flowchart showing an example of a voice dialogue processing procedure by the voice dialogue device 300 according to the first embodiment. As shown in FIG. 5, the voice dialogue device 300 inputs the dialogue voice of the user 101, executes the voice recognition process by the voice recognition unit 401, and outputs the voice recognition result 500 (step S1001).

Next, the voice dialogue device 300 acquires the reliability of the word string Wt and the reliability of the words W1 to Wn constituting the word string Wt by the reliability acquisition unit 403 as shown in FIG. (Step S1002). Further, the voice dialogue device 300 acquires the language likelihood for each language model by the language likelihood acquisition unit 402 (step S1003).

When the dialogue context language model 415 is used in step S1003, the language likelihood acquisition unit 402 connects a sentence (word) in which the voice dialogue device 300 before the spoken voice 110 of the user 101 and the spoken voice 110 of the user 101 are connected. Get the language likelihood by taking the column) as an input.

for example,
Voice Dialogue Device 300: "Hello"
User 101: "Hello, your name is"
Voice Dialogue Device 300: "My name is a robot"
User 101: "Hey, that's cute, isn't it?"
Voice dialogue device 300: "Thank you"
User 101: "What can you do?" (Voice voice 110)
Take the dialogue as an example.

In this case, the voice "hello" of the voice dialogue device 300 before the voice 110 of the user 101 "what can you do?", The voice "hello, your name" of the user 101, the voice. Obtained from the voice recognition unit 401 about the voice of the dialogue device 300 "My name is robot 102", the voice of the user 101 "Hey, that's cute", and the voice of the voice dialogue device 300 "Thank you". Connect the word strings together to make the word string "Hello. Hello, your name is. My name is a robot. Oh yeah, cute. Thank you. What can you do?"

The voice dialogue device 300 uses this connected word string "Hello. Hello, your name. My name is a robot. Oh yeah, cute. Thank you. What can you do?" Input to the language model 415 to calculate the language likelihood. In this way, by inputting not only the spoken voice 110 of the user 101 but also the dialogue before that into the dialogue context language model 415, it is possible to improve the accuracy of the language likelihood obtained from the dialogue context language model 415. ..

When the dialogue context language model 415 is used in step S1003, the language likelihood acquisition unit 402 connects each word of the word string Wt of the speech recognition result 500 to generate a sentence, and the morphology element possessed by the dialogue context language model 415. The word string Wt'may be used in the morphological analysis of the analyzer. As a result, the word delimiter may differ between the word string Wt and the word string Wt'. Then, the language likelihood acquisition unit 402 calculates the likelihood with the word string Wt'.

Then, the voice dialogue device 300 determines whether or not to listen back to the user 101 by the listening back determination unit 404 (step S1004). Specifically, for example, the voice dialogue device 300 determines whether or not any one of the language likelihood for each language model and the reliability of the word string Wt is less than the threshold value. When any one of the language likelihood for each language model and the reliability of the word string Wt is less than the threshold value (step S1004: Yes), that is, when the user 101 should be heard back, the voice dialogue device. In the 300, the dialogue process is executed by the dialogue generation unit 406 (step S1005).

The voice dialogue device 300 generates the response sentence of any of the listening back patterns P1 to P5 by the listening back sentence generation process (step S1005), or notifies that the response sentence is not generated, and proceeds to step S1009. The details of the back-to-back sentence generation process (step S1005) will be described later with reference to FIG.

On the other hand, when neither the language likelihood for each language model nor the reliability of the word string Wt is less than the threshold value (step S1005: No), that is, when there is no need to listen back to the user 101, the voice dialogue device 300 , It is determined whether or not the user 101 is being heard back by the back-listening pattern P1, P3, or P4 (step S1006).

When the listening pattern P1, P3, or P4 is being heard back (step S1006: Yes), the voice dialogue device 300 executes the rephrasing utterance interpretation process by the hearing back sentence generation unit 406 (step S1007), and steps S1009. Move to. The rephrasing utterance interpretation process (step S1007) is a process of interpreting the utterance voice rephrased by the user 101 and responding according to the interpretation. The details of the rephrasing interpretation process (step S1007) will be described later with reference to FIG.

On the other hand, when the listening pattern P1, P3, or P4 is not being heard back (step S1006: No), the voice dialogue device 300 executes the dialogue control process by the dialogue control unit 405 (step S1008), and proceeds to step S1009. .. The dialogue control process (step S1008) is a process of generating a response sentence in response to the user 101 utterance voice. The details of the dialogue control process (step S1008) will be described later with reference to FIG.

In step S1009, the dialogue history management unit 407 uses the dialogue history management unit 407 to input the word string Wt of the user 101 utterance voice, the listening back process (step S1005), the rephrasing interpretation process (step S1007), and the dialogue control process (step S1008). ) Is accumulated as a dialogue history (step S1009).

Then, the voice dialogue device 300 converts the response sentence generated by the dialogue control unit 405 or the listening back sentence generation unit 406 by the voice synthesis unit 408 into voice and outputs it (step S1010). When the voice dialogue device 300 is realized as a stand-alone type, the voice dialogue device 300 executes the processes of steps S1001 to S1010 shown in FIG.

On the other hand, when the voice dialogue device 300 is realized by the server 201 of the client server system, the voice dialogue device 300 executes steps S1001 to 1009 and sends a response message to a communication device such as a communication robot 102 or a smartphone which is a client. Send. Then, the client converts the response sentence generated by the dialogue control unit 405 or the listening back sentence generation unit 406 by the voice synthesis unit 408 into voice and outputs it (step S1010).

<Return sentence generation process (step S1005)>
FIG. 11 is a flowchart showing a detailed processing procedure example of the hearing back sentence generation processing (step S1005) shown in FIG. The hearing back sentence generation unit 406 determines whether or not the reliability of the word string Wt is less than the threshold value (step S1101).

When the reliability of the word string Wt is less than the threshold value (step S1102: Yes), the response sentence generation unit 406 determines whether or not the total reliability of the words W1 to Wn is less than the threshold value (step S1102: Yes). Step S1102). When the total reliability of the words W1 to Wn is less than the threshold value (step S1102: Yes), the response sentence generation unit 406 has the reliability of any part of the words W1 to Wn less than the threshold value. (Step S1103).

In the case of part A (step S1103: part A), the process proceeds to step S1104, and in the case of part B (step S1103: part B), the process proceeds to step S1105, and in the case of part C (step S1103: part C), step S1106. Move to. As an example, when the word string Wt corresponds to both the part A and the part B, the part A is preferentially applied, and when the word string Wt corresponds to both the part B and the part C, the part B is preferentially applied.

The part A is a plurality of words existing in the first half or the second half of the word string Wt and having a reliability of less than the threshold value. If the number of words n of the words W1 to Wn is even, the first half is the words W1 to W (n / 2) among the words W1 to Wn, and the second half is the word W (n / 2 + 1) among the words W1 to Wn. ) ~ Wn. If the number of words n of the words W1 to Wn is odd, the first half is the words W1 to W ((n + 1) / 2) among the words W1 to Wn, and the latter half is the word W (((n + 1) / 2) among the words W1 to Wn. n + 1) / 2) to Wn.

However, the plurality of words include at least one independent word. An independent word is a word of a part of a word (verb, adjective, adjective verb, noun, adverb, adnominal adjective, connective verb, emotional verb) other than an adverb. An adjunct is a word whose part of speech is an auxiliary verb or particle. In the case of the part A (step S1103: part A), there is a word group in the first half or the second half of the spoken voice of the user 101, which is difficult for the voice dialogue device 300 to recognize the voice. That is, the part A corresponds to the case where the voice dialogue device 300 cannot hear half of the spoken voice of the user 101.

In the case of the part A (step S1103: part A), the back sentence generation unit 406 listens back to the remaining part of the word string Wt excluding the part, that is, the back sentence for listening back the word whose reliability is equal to or higher than the threshold value. It is generated as P1 (see FIG. 1) (step S1104), and the process proceeds to step S1009.

Further, the portion B is a plurality of words (including at least one independent word) that exist discretely in the word string Wt and whose reliability is less than the threshold value. However, in order to avoid duplication with the part A, it is necessary that at least one word having a reliability lower than the threshold value exists in each of the first half and the second half of the word string Wt. That is, the part B corresponds to the case where the voice dialogue device 300 has a part of the spoken voice 110 of the user 101 that cannot be heard in fragments.

In the case of the part B (step S1103: part B), a back sentence for listening back a word whose reliability does not correspond to the part B in the word string Wt is generated as a back pattern P2 (see FIG. 1) (step S1104). ), The process proceeds to step S1009.

Further, the partial C is one independent word whose reliability is less than the threshold value in the word string Wt, or two consecutive words (independent words) whose reliability is less than the threshold value in the word string Wt. Including at least one). However, if it overlaps with the part A, the part A has priority (the part C may be applied preferentially over the part A). That is, the portion C corresponds to the case where the voice dialogue device 300 cannot hear a part of the spoken voice 110 of the user 101.

In the case of the part C (step S1103: part C), the hearing back sentence generation unit 406 executes the mask word estimation process (step S1106). The mask word estimation process (step S1106) is a process of estimating a mask word. The mask word is a word corresponding to the part C. The details of the mask word estimation process (step S1106) will be described later with reference to FIG.

After executing the mask word estimation process (step S1106), the listening back sentence generation unit 406 determines whether or not the mask word estimation is successful (step S1107). When the estimation of the mask word is successful (step S1107: Yes), the hearing back sentence generation unit 406 outputs the estimation result of the portion C whose reliability is less than the threshold value and the utterance voice 110 of the user 101 including the estimation result. The utterance sentence to be confirmed is generated as the utterance pattern P3 (see FIG. 1).

In the case of the back-to-back pattern P3 in FIG. 1, the estimation result of the portion C is "Mt. Fuji", and the back-to-back sentence confirming the utterance voice 110 of the user 101 including the estimation result is "Tomorrow's sunrise time of Mt. Fuji". Is.

On the other hand, when the estimation of the mask word is not successful (step S1107: No), the hearing-back sentence generation unit 406 requests a re-question, that is, the back-listening sentence that re-requests the spoken voice 110 is heard back pattern P5 (FIG. 1). (See) (see step S1109), and the process proceeds to step S1009.

Further, in step S1102, even when the reliability of all words W1 to Wn is less than the threshold value (step S1102: Yes), the hearing back sentence generation unit 406 executes step S1109 and proceeds to step S1009.

Further, in step S1101, when the reliability of the word string Wt is equal to or higher than the threshold value (step S1102: No), whether the verbal likelihood of the standard language model 412 is less than the threshold value in the response sentence generation unit 406. It is determined whether or not (step S1110). If the language likelihood of the standard language model 412 is less than the threshold value (step S1110: Yes), step S1109 is executed and the process proceeds to step S1009.

On the other hand, when the language likelihood of the standard language model 412 is equal to or greater than the threshold value (step S1110: No), the response sentence generation unit 406 determines whether or not the language likelihood of the question sentence language model 414 is less than the threshold value. (Step S1111).

When the language likelihood of the question sentence language model 414 is equal to or greater than the threshold value (step S1111: No), the language likelihood of the field-specific language model 413 or the dialogue context language model 415 is less than the threshold value. Therefore, the re-listening sentence generation unit 406 generates a re-listening sentence for reciting and confirming the entire question (spoken voice 110 of the user 101) as a re-listening pattern P4 (see FIG. 1) (step S1112), and proceeds to step S1009. .. Further, even when neither the field-specific language model 413 nor the dialogue context language model 415 is used, when the language likelihood of the question sentence language model 414 is equal to or greater than the threshold value (step S1111: No), the response sentence is generated. Section 406 executes step S1112.

On the other hand, when the language likelihood of the question sentence language model 414 is less than the threshold value (step S1111: Yes), the listening back sentence generation unit 406 does not generate the listening back sentence and notifies that the listening back sentence is not generated (step S1111: Yes). Step S1113) and step S1009. In this way, the listening back sentence generation unit 406 can generate the listening back sentences of the listening back patterns P1 to P5 according to the reliability of each word and the language likelihood of each language model.

<Mask word estimation process (step S1106)>
FIG. 12 is a flowchart showing a detailed processing procedure example of the mask word estimation process (step S1106) shown in FIG. FIG. 13 is an explanatory diagram showing an example of the mask word estimation process (step S1106) shown in FIG. The mask word estimation process (step S1106) is executed in the case of step S1103: partial C. In FIG. 13, the word string Wt including the partial C “○ × ▽” is defined as the word string 1300 “tell me the height of ○ × ▽”.

The hearing-back sentence generation unit 406 extracts the reading of the word "○ × ▽" which is the part C (step S1201). In FIG. 13, it is assumed that "Fusan" is extracted as the reading 1302 of "○ × ▽". Next, the listening back sentence generation unit 406 masks the word "○ × ▽" in the part C to make it "***" (step S1202). The word string 1300 after the mask of the word "○ × ▽" in the part C is set as the word string 1301.

Next, the listening back sentence generation unit 406 inputs the masked word string 1301 into the BERT 1310, which is an example of the dialogue context language model 415, and predicts the masked word (step S1203). Here, as the prediction result 1303, "Landmark Tower" (prediction mask word 1303A), "Tokyo Tower" (prediction mask word 1303B), and "Mt. Fuji" (prediction mask word 1303C) are predicted.

Next, the listening back sentence generation unit 406 extracts the readings of the prediction mask words 1303A to 1303C (step S1204). Here, as the extraction result 1304, "Landmark Tower" (predictive mask word reading 1304A), "Tokyo Tower" (predictive mask word reading 1304B), and "Fujisan" (predictive mask word reading 1304C) are predicted.

Then, the hearing-back sentence generation unit 406 edits each of the readings 1302 of the word "○ × ▽" of the part C, “Fusan”, and the readings of the prediction mask words 1304A to 1304C, for example, the editing distance (Levenshtein distance). (Step S1206). The hearing-back sentence generation unit 406 determines whether or not there is a reading 1302 of the word “○ × ▽” in the portion C and readings 1304A to 1304C of the predicted mask words within a predetermined distance (step S1206).

When there is no reading 1304A to 1304C of the predicted mask word within a predetermined distance (step S1206: No), the listening back sentence generation unit 406 ends the mask word estimation process (step S1106) and proceeds to step S1107. In this case, in step S1107, the mask word estimation fails (step S1107: No).

On the other hand, when there are readings 1304A to 1304C of the prediction mask word within a predetermined distance (step S1206: Yes), the listening back sentence generation unit 406 selects the prediction mask word with the shortest editing distance as the prediction mask word with the closest reading. (Step S1207). Here, as an example, "Mt. Fuji" (predictive mask word 1303C) whose reading is "Fujisan" (predictive mask word reading 1304C) is selected. Then, the back-to-back sentence generation unit 406 ends the mask word estimation process (step S1106), and proceeds to step S1107. In this case, in step S1107, the mask word estimation is successful (step S1107: Yes).

<Rephrasing utterance interpretation processing (step S1007)>
FIG. 14 is a flowchart showing a detailed processing procedure example of the rephrasing utterance interpretation processing (step S1007) shown in FIG. The rephrasing utterance interpretation process (step S1007) is executed in the case of step S1006: Yes, that is, in the case of listening back in the listening back pattern P1, P3 or P4.

In the case of step S1006: Yes, if the listening back pattern during listening back is P1 (step S1401: P1), the listening back sentence generation unit 406 rephrases the spoken voice of the user 101 (word string Wt of the spoken voice input this time). And the recognition result of the previous utterance are combined (step S1402), and the process proceeds to the dialogue control process (step S1403).

Here, it is assumed that the recognition result of the previous utterance, which is the listening pattern P1, is "tomorrow's sunrise of Mt. Fuji". "Tomorrow's sunrise of Mt. Fuji" is a group of words whose reliability of each word is above the threshold.

When "Tomorrow's sunrise of Mt. Fuji" is the first half of the word string of the previous spoken voice of the user 101, the rehearsal sentence generation unit 406 uses the rephrased voice of the user 101 (the word string of the spoken voice input this time). Wt (for example, "tell me the time") is concatenated at the end of "tomorrow's sunrise on Mt. Fuji" to generate "tell me the time of tomorrow's sunrise on Mt. Fuji".

Further, it is assumed that the recognition result of the previous utterance, which is the listening pattern P1, is "Tell me the time of sunrise". "Tell me the time of sunrise" is a group of words whose reliability of each word is equal to or higher than the threshold value.

When "Tell me the time of sunrise" is the first half of the word string of the utterance voice of the previous user 101, the rehearsal sentence generation unit 406 uses the rephrased utterance voice of the user 101 (the word string Wt of the utterance voice input this time). (For example, "Tomorrow's Mt. Fuji") is connected to the beginning of "Tell me the time of sunrise" to generate "Tell me the time of sunrise tomorrow's Mt. Fuji".

However, the rehearsal sentence generation unit 406 deletes one of the parts that match the rephrased utterance voice of the user 101 (word string Wt of the utterance voice input this time) and the recognition result of the previous utterance. Prevent redundancy.

For example, if "tell me the time of sunrise" is the recognition result of the previous utterance and the word string of the utterance voice of the previous user 101 is "the time of sunrise of Mt. Fuji tomorrow", the combined result is "tomorrow". The time of sunrise on Mt. Fuji Please tell me the time of sunrise. " In this case, since the "sunrise time" appears twice, the hearing-back sentence generation unit 406 deletes one "sunrise time" to "tell me the sunrise time of Mt. Fuji tomorrow".

Further, if the listening back pattern during listening back is P3 or P4 (step S1401: P3, P4), the listening back sentence generation unit 406 tells us that the listening back pattern is P3 or P4 (example: "Tell me the height of Mt. Fuji". Did you? ”), It is determined whether the answer of the user 101 (word string Wt of the spoken voice input this time) is affirmative or negative (step S1404). If it is affirmative (step S1404: affirmative), the process proceeds to the dialogue control process (step S1403). The dialogue control process (step S1403) will be described later with reference to FIG.

On the other hand, if it is negative (step S1404: negative), the response sentence generation unit 406 generates a response sentence requesting a re-question (example: "Please ask the question again") (step S1406), and step S1007. Move to.

<Dialogue control processing (steps S1008, S1403)>
FIG. 15 is a flowchart showing a detailed processing procedure example of the dialogue control processing (steps S1008 and S1403) shown in FIGS. 10 and 14. First, the dialogue control unit 405 searches for a hypothetical question sentence close to the word string Wt of the voice recognition result 500 with reference to the dialogue knowledge DB 416 (step S1501). Specifically, for example, the dialogue control unit 405 calculates the similarity with the word string Wt for each assumed question sentence by the editing distance between the word string Wt and the assumed question sentence of the dialogue knowledge DB 416. Here, as an example, the reciprocal of the editing distance is used as the degree of similarity. Therefore, the larger the similarity value is, the more similar the assumed question sentence is to the word string Wt.

Next, the dialogue control unit 405 determines whether or not there is an assumed question sentence whose similarity is equal to or higher than the threshold value (step S1502). When there is no assumed question sentence whose similarity is equal to or higher than the threshold value (step S1503: No), the dialogue control unit 405 generates a response sentence indicating that the meaning of the question is not understood, and proceeds to steps S1009 and S1007. On the other hand, when there is an assumed question sentence whose similarity is equal to or higher than the threshold value (step S1502: Yes), the dialogue control unit 405 provides an answer sentence corresponding to the assumed question sentence whose similarity is equal to or higher than the threshold value in the dialogue knowledge DB 416. It is output as a response statement (step S1504), and the process proceeds to steps S1009 and S1007.

<Dialogue example>
FIG. 16 is a flowchart showing an example of the flow of dialogue between the user 101 and the voice dialogue device 300. It is assumed that the user 101 utters "Tell me the height of Mt. Fuji" (step S1601). When the voice dialogue device 300 recognizes "tell me the height of Mt. Fuji" as "tell me the height of ***" (step S1602), it estimates "***" of the mask portion and "Tells me the height of Mt. Fuji". Is it Mt. Fuji? ”(Step S1603) (listening pattern P3).

When the user 101 responds with "No" meaning denial to "Mt. Fuji?" (Step S1604), the voice dialogue device 300 is denied "Mt. Fuji" which is the estimation result of the mask portion. Say "Please ask the question again" (step S1605).

Further, when the user 101 responds to the question "Is it Mt. Fuji?" In step S1603 with "Yes" meaning affirmation (step S1606), the voice dialogue device 300 speaks the utterance of the user 101 in S1601 to "Mt. Fuji". Please tell me the height of. " Therefore, the voice dialogue device 300 searches for the corresponding answer "3776 meters" from the dialogue knowledge DB 416 (step S1610) with "the height of Mt. Fuji" as the assumed question sentence, and "the height of Mt. Fuji is 3776 meters." (Step S1611).

Further, when the voice dialogue device 300 recognizes "Mt. Fuji ***" in response to "Tell me the height of Mt. Fuji" which is the utterance voice 110 of the user 101 in step S1601 (step S1607), "Tell me the height of Mt. Fuji". What is it? ”(Step S1608) (listening pattern P1). On the other hand, when the user 101 responds with "height" (step S1609), the voice dialogue device 300 searches the dialogue knowledge DB 416 for the corresponding answer "3776 meters" with "the height of Mt. Fuji" as the assumed question sentence. (Step S1610), he says, "The height of Mt. Fuji is 3776 meters." (Step S1611).

As described above, according to the first embodiment, the listening back patterns P1 to P5 for the spoken voice 110 of the user 101 can be selected according to the reliability of the word and the language likelihood. In particular, by evaluating the utterance voice 110 of the user 101 using the standard language model 412, the field-specific language model 413, the question sentence language model 414, and the dialogue context language model 415, each of them is plausible as Japanese. It is possible to judge whether it is plausible as an utterance in the field, plausible as a question, or plausible after considering the dialogue context, and by listening back based on the judgment result, it is possible to prevent the dialogue from breaking down. Can be done.

In this way, by performing the minimum necessary re-listening based on the reliability of voice recognition, it is possible to prevent the user 101 from asking the same question over and over, and it is possible to reduce the annoyance of the user 101. can.

In the second embodiment, in the case where the language likelihood of the user 101 is equal to or higher than the threshold value according to the field-specific language model 413 or the dialogue context-specific language model due to the unique wording of the spoken voice 110 of the user 101. On the other hand, this is an example of preventing an erroneous determination, that is, a transition to step S1004: No. As a result, the voice dialogue device 300 requests the user 101 to re-question (step S1405), or responds by the dialogue control process (steps S1008, S1403) that the meaning of the assumed question sentence is not understood (step S1503). , It is prevented from giving an unprobable answer corresponding to an unprobable assumed question sentence (step S1504). In addition, since the content of the second embodiment will be mainly described here, the description of the part overlapping with the first embodiment will be omitted.

<Example of functional configuration of voice dialogue device 300>
FIG. 17 is a block diagram showing a functional configuration example of the voice dialogue device 300 according to the second embodiment. In addition to the configuration shown in FIG. 4 of the first embodiment, the voice dialogue device 300 has an individual language model 1701 and an individual identification unit 1702. The personal language model 1701 is stored in the storage device 302 of the voice dialogue device 300 shown in FIG. 3 or the storage device 302 of another computer capable of communicating with the voice dialogue device 300. Specifically, the personal identification unit 1702 is realized, for example, by causing the processor 301 to execute a voice dialogue program stored in the storage device 302 shown in FIG.

The personality language model 1701 is a language model created by the property (individuality) peculiar to the user 101, and specifically, for example, is created for each user 101 from the dialogue history of each user 101. Therefore, the dialogue history management unit 407 manages the dialogue history for each user 101. The personal language model 1701 is realized by, for example, the words N-gram and RNN.

The personal identification unit 1702 identifies the user 101. Specifically, the personal identification unit 1702 manages the biometric information of the user 101 such as fingerprints, veins of the palm, iris, facial image, and voice, and when it matches the input biometric information, the input person is selected. Identify as user 101 with biometric information. Further, the personal identification unit 1702 manages the user ID and password, and may identify the input person as the user 101 when the user ID and password match the input user ID and password.

<Example of voice dialogue processing procedure>
FIG. 18 is a flowchart showing an example of a voice dialogue processing procedure by the voice dialogue device 300 according to the second embodiment. The voice dialogue device 300 executes steps S1801 to S1804 prior to steps S1001 to S1010 shown in FIG.

Specifically, for example, the voice dialogue device 300 identifies the user 101 who has input the biometric information by the personal identification unit 1702 (step S1801). The voice dialogue device 300 determines whether or not the user 101 who has input the biometric information is the registered user 101 (step S1802). Specifically, for example, the voice dialogue device 300 determines whether or not the registered biometric information and the input biometric information match, and if they match, the user 101 who has input the biometric information is registered. It is determined that the user is 101.

When the user is a registered user 101 (step S1802: Yes), the voice dialogue device 300 loads the personal language model 1701 of the user 101 and the dialogue history (step S1803), and proceeds to step S1001. On the other hand, when the user is not the registered user 101 (step S1802: No), the voice dialogue device 300 newly creates the personal language model 1701 and the dialogue history of the user 101 (hereinafter, new user 101) (step S1804). .. Specifically, for example, the voice dialogue device 300 interacts with the new user 101 to acquire a dialogue history, and creates a personal language model 1701 of the new user 101 based on the acquired dialogue history. Then, the process proceeds to step S1001.

The personal language model 1701 is used in the language likelihood calculation (step S1003). When the language likelihood of the question sentence language model 414 is equal to or greater than the threshold value in step S1111 shown in FIG. 11 (step S1111: No), the field-specific language model 413, the dialogue context language model 415, or the individual language model 1701 The language likelihood of any of is less than the threshold. Therefore, the re-listening sentence generation unit 406 generates a re-listening sentence for reciting and confirming the entire question (spoken voice of the user 101) as a re-listening pattern P4 (see FIG. 1) (step S1112), and proceeds to step S1009.

Further, even when none of the field-specific language model 413, the dialogue context language model 415, and the individual language model 1701 is used, the language likelihood of the question sentence language model 414 is equal to or higher than the threshold value (step S1111:). No), the listening back sentence generation unit 406 executes step S1112.

As described above, according to the second embodiment, it is possible to suppress the erroneous determination of the language likelihood caused by the wording peculiar to the user 101 and to facilitate the dialogue.

Further, the voice dialogue device 300 according to the first and second embodiments described above can be configured as described in (1) to (13) below.

(1) In the voice dialogue device 300 having a processor 301 for executing the voice dialogue program and a storage device 302 for storing the voice dialogue program, the processor 301 constitutes each word W1 to form a word string Wt relating to the spoken voice 110. Based on the reliability acquisition process (step S1102) for acquiring the reliability of Wn and the reliability acquired by the reliability acquisition process (step S1102), a response sentence to be heard back to the user 101 who is the utterance source of the utterance voice 110 is sent back. The process of generating a back-to-back sentence (step S1005) to be generated is executed.

As a result, the voice dialogue device 300 can generate an appropriate return sentence from the return patterns P1 to P3 and P5 as shown in FIG. 11, for example.

(2) In the voice dialogue device 300 of the above (1), in the utterance sentence generation process (step S1005), the processor 301 has a first threshold value for the reliability of all the words W1 to Wn constituting the word string Wt. If it is less than (step S1102: Yes), a back sentence for re-requesting the spoken voice 110 is generated (step S1109).

As a result, the voice dialogue device 300 can, for example, generate a back-to-back sentence of the back-to-back pattern P5 from the back-to-back patterns P1 to P3 and P5 as shown in FIG. 11 and request the user 101 to re-question. ..

(3) In the voice dialogue device 300 of the above (1), in the back-to-back sentence generation process (step S1005), the processor 301 has a word string Wt in which a word having a reliability equal to or higher than the first threshold value is present (step S1102). : No), a back sentence is generated based on the position of a word whose reliability is less than the first threshold value in the word string Wt.

As a result, the voice dialogue device 300 generates, for example, a return sentence according to the position of a word whose reliability is less than the first threshold value from the return patterns P1 to P3 and P5 as shown in FIG. be able to.

(4) In the voice dialogue device 300 of the above (3), in the back-to-back sentence generation process (step S1005), the processor 301 has a word whose reliability is less than the first threshold value in the first half or the second half of the word string Wt. In the case (step S1103: part A), a dialogue sentence is generated to listen back to the first half or the second half in which the word whose reliability is less than the first threshold value does not exist (step S1104).

As a result, the voice dialogue device 300 generates, for example, a back-to-back sentence of the back-to-back pattern P1 from the back-to-back patterns P1 to P3 and P5 as shown in FIG. 11, and the user 101 can hear the spoken voice 110. It is possible to utter the part that is present and encourage the recurrence of the part that cannot be heard.

(5) In the voice dialogue device 300 of the above (3), in the back-to-back sentence generation process (step S1005), the processor 301 has a plurality of words whose reliability is less than the first threshold value over the first half and the second half of the word string Wt. (Step S1103: Part B), a back sentence is generated to listen back to the word having the first threshold value or more in the word string Wt (step S1105).

As a result, the voice dialogue device 300 generates, for example, a back-to-back sentence of the back-to-back pattern P2 from the back-to-back patterns P1 to P3 and P5 as shown in FIG. 11, and the user 101 can hear the spoken voice 110. You can ask the user 101 to ask a question again by speaking the part.

(6) In the voice dialogue device 300 of (3) above, the processor 301 uses one word whose reliability is less than the first threshold value, or two consecutive words whose reliability is less than the first threshold value. Is present in the word sequence (step S1103: part C), one word is based on the dialogue context language model 415 trained in the statistics of the word group appearing in and around any word and the statistics of the sequence of phrases. Alternatively, in the mask word estimation process (step S1106) for estimating what kind of word the two consecutive words are, and in the listening back sentence generation process (step S1005), the processor 301 performs the mask word estimation process (step). Generates a reply sentence according to the estimation result by S1106).

Thereby, the voice dialogue device 300 estimates from the context, for example, one word whose reliability is less than the first threshold value or two consecutive words whose reliability is less than the first threshold value. , It is possible to reduce the frequency of listening back.

(7) In the voice dialogue device 300 of the above (6), in the hearing back sentence generation process (step S1005), the processor 301 estimates when the estimation by the mask word estimation process (step S1106) is successful (step S1107: Yes). A dialogue sentence is generated (step S1108), which includes the words and confirms the spoken voice 110.

As a result, the voice dialogue device 300 can generate a back-to-back sentence confirming the estimated word and the entire spoken voice 110 as the back-to-back pattern P3, for example, which part was difficult to recognize, and as a whole, It is possible to inform the user 101 how the recognition was made and reduce the frequency of listening back.

(8) In the voice dialogue device 300 of the above (6), in the back-to-back sentence generation process (step S1005), the processor 301 speaks when the estimation by the mask word estimation process (step S1106) fails (step S1107: No). Generate a dialogue that re-requests the voice 110.

As a result, the voice dialogue device 300 can generate, for example, a back-to-back sentence of the back-to-back pattern P5 and request the user 101 to re-question.

(9) In the voice dialogue device 300 of the above (1), in the back-to-back sentence generation process (step S1005), the processor 301 has a case where the reliability of the word string Wt is less than the first threshold value (step S1101: Yes). , A back sentence is generated based on the reliability of each word W1 to Wn constituting the word string Wt.

As a result, if the reliability of the word string Wt is less than the first threshold value, it is assumed that the spoken voice 110 is not correctly recognized, and the reliability of the individual words W1 to Wn is used to determine how to listen back. It can be selected from P1 to P3 and P5.

(10) In the voice dialogue device 300 of the above (1), the processor 301 acquires a plurality of language likelihood obtained as a result of inputting a word string into each of the plurality of language models 421-415 (language likelihood acquisition process). In step S1103) and the response sentence generation process (step S1005), the processor 301 performs the language likelihood acquisition process (step S1101: No) when the reliability of the word string Wt is equal to or higher than the first threshold value (step S1101: No). A back sentence is generated based on the plurality of language likelihood acquired in step S1103).

Thereby, when the recognition of the spoken voice 110 is highly reliable, it is possible to specify which language model the word string Wt is likely to be spoken by, and to select the listening pattern P4, P5, or no listening. ..

(11) In the voice dialogue device 300 of the above (10), in the utterance sentence generation process (step S1005), the processor 301 is a sequence of words obtained from texts in a plurality of fields among a plurality of language models 421-415. When the first language likelihood obtained as a result of inputting the word string Wt into the standard language model 412, which is a statistical model, is less than the second threshold value (step S1110: Yes), the spoken voice 110 is re-requested. Generate a statement (step S1109).

As a result, when it is not plausible as an utterance in a standard language, it is possible to generate a back-to-back sentence of the back-to-back pattern P5 and prompt the user 101 to re-question.

(12) In the voice dialogue device 300 of the above (11), in the hearing back sentence generation process (step S1005), when the first language likelihood is equal to or higher than the second threshold value (step S1110: Yes), the processor 301 The second language likelihood obtained as a result of inputting the word string Wt into the question sentence language model 414 that determines whether or not the spoken voice 110 is a question from the sequence of words constituting the question sentence is less than the second threshold value. If (step S1111: Yes), the utterance sentence is not generated (step S1113).

As a result, when the question sentence is considered to be plausible as a question sentence in the question sentence language model 414, it is possible to suppress unnecessary repetition of listening back and promote smooth dialogue.

(13) In the voice dialogue device 300 of the above (11), in the hearing back sentence generation process (step S1005), the processor 301 constitutes a question sentence when the first language likelihood is equal to or more than the second threshold value. If the second language likelihood obtained as a result of inputting the word string Wt into the question sentence language model 414 that determines whether or not the spoken voice 110 is a question is equal to or higher than the second threshold value (step S1111). : No), a return sentence for listening back to the spoken voice 110 is generated (step S1112).

As a result, even if the spoken voice 110 is voice-recognized, if the question sentence language model 414 determines that the question sentence is not plausible, the question-back sentence of the question-back pattern P4 can be generated and the entire question can be requested again. can.

It should be noted that the present invention is not limited to the above-mentioned examples, but includes various modifications and equivalent configurations within the scope of the attached claims. For example, the above-described embodiment has been described in detail in order to explain the present invention in an easy-to-understand manner, and the present invention is not necessarily limited to those having all the described configurations. Further, a part of the configuration of one embodiment may be replaced with the configuration of another embodiment. Further, the configuration of another embodiment may be added to the configuration of one embodiment. In addition, other configurations may be added, deleted, or replaced with respect to a part of the configurations of each embodiment.

Further, each of the above-mentioned configurations, functions, processing units, processing means, etc. may be realized by hardware by designing a part or all of them by, for example, an integrated circuit, and the processor 301 performs each function. It may be realized by software by interpreting and executing the program to be realized.

Information such as programs, tables, and files that realize each function is recorded in a storage device such as a memory, a hard disk, an SSD (Solid State Drive), or an IC (Integrated Circuit) card, an SD card, or a DVD (Digital Versail Disc). It can be stored in a medium.

In addition, the control lines and information lines show what is considered necessary for explanation, and do not necessarily show all the control lines and information lines necessary for mounting. In practice, it can be considered that almost all configurations are interconnected.

W1 to Wn Word WL Word Lattice Wt Word string 101 User 102 Communication robot 103 Smartphone 104 Information processing device 110 Speech voice 200 Voice dialogue system 201 Server 202 Network 300 Voice dialogue device 301 Processor 302 Storage device 401 Voice recognition unit 402 Language likelihood acquisition Part 403 Reliability acquisition part 404 Hearing back judgment part 405 Dialogue control part 406 Hearing back sentence generation part 407 Dialogue history management part 408 Speech synthesis part 411 Speech recognition model 412 Standard language model 413 Field-specific language model 414 Questionnaire language model 415 Dialogue context language Model 416 Dialogue knowledge DB

Claims (15)

A voice dialogue device comprising a processor that executes a program and a storage device that stores the program.
The processor
The acquisition process to acquire the reliability of each word that composes the word string related to the spoken voice, and
Based on the reliability acquired by the acquisition process, a generation process for generating a back sentence to be heard back to the originator of the uttered voice, and a generation process.
A voice dialogue device characterized by performing. The voice dialogue device according to claim 1.
In the generation process, the processor generates a back sentence that re-requests the spoken voice when the reliability of all the words constituting the word string is less than the first threshold value.
A voice dialogue device characterized by that. The voice dialogue device according to claim 1.
In the generation process, the processor is based on the position of the word having the reliability less than the first threshold value in the word string when the word having the reliability equal to or higher than the first threshold value is present in the word string. To generate the above-mentioned return sentence,
A voice dialogue device characterized by that. The voice dialogue device according to claim 3.
In the generation process, when a word whose reliability is less than the first threshold value is present in the first half or the second half of the word string, the processor has the first reliability among the first half or the second half. Generate a reply sentence that listens back to the person who does not have a word less than the threshold value,
A voice dialogue device characterized by that. The voice dialogue device according to claim 3.
In the generation process, when the processor has a plurality of words whose reliability is less than the first threshold value over the first half and the second half of the word string, the word of the word string having the first threshold value or more is present. Generate a reply sentence,
A voice dialogue device characterized by that. The voice dialogue device according to claim 3.
The processor
If one word whose reliability is less than the first threshold value or two consecutive words whose reliability is less than the first threshold value are present in the word string, any word and its word are present. Estimating what kind of word the one word or two consecutive words are based on the dialogue context language model trained by the statistics of the word group appearing in the surroundings and the statistics of the sequence of phrases. Execute the process,
In the generation process, the processor generates a back sentence according to the estimation result by the estimation process.
A voice dialogue device characterized by that. The voice dialogue device according to claim 6.
In the generation process, when the estimation by the estimation process is successful, the processor generates a back sentence including the estimated word and confirming the spoken voice.
A voice dialogue device characterized by that. The voice dialogue device according to claim 6.
In the generation process, the processor generates a back sentence that re-requests the spoken voice when the estimation by the estimation process fails.
A voice dialogue device characterized by that. The voice dialogue device according to claim 1.
In the generation process, when the reliability of the word string is less than the first threshold value, the processor generates the back sentence based on the reliability of each word constituting the word string.
A voice dialogue device characterized by that. The voice dialogue device according to claim 1.
In the acquisition process, the processor acquires a plurality of language likelihoods obtained as a result of inputting the word string into each of the plurality of language models.
In the generation process, when the reliability of the word string is equal to or higher than the first threshold value, the processor generates the return sentence based on the plurality of language likelihoods acquired by the acquisition process.
A voice dialogue device characterized by that. The voice dialogue device according to claim 10.
In the generation process, the processor inputs the word string into the first language model, which is a statistical model of a sequence of words obtained from texts in a plurality of fields among the plurality of language models. If the language likelihood is less than the second threshold, a response sentence reclaiming the spoken voice is generated.
A voice dialogue device characterized by that. The voice dialogue device according to claim 11.
In the generation process, when the first language likelihood is equal to or higher than the second threshold value, the processor determines from the sequence of words constituting the question sentence whether or not the spoken voice is a question. If the second language likelihood obtained as a result of inputting the word string into the language model is less than the second threshold value, the utterance sentence is not generated.
A voice dialogue device characterized by that. The voice dialogue device according to claim 11.
In the generation process, when the first language likelihood is equal to or higher than the second threshold value, the processor determines from the sequence of words constituting the question sentence whether or not the spoken voice is a question. If the second language likelihood obtained as a result of inputting the word string into the language model is equal to or greater than the second threshold value, a back sentence that listens back to the spoken voice is generated.
A voice dialogue device characterized by that. A voice dialogue method executed by a voice dialogue device having a processor for executing a program and a storage device for storing the program.
The voice dialogue method is
The processor
The acquisition process to acquire the reliability of each word that composes the word string related to the spoken voice, and
Based on the reliability acquired by the acquisition process, a generation process for generating a back sentence to be heard back to the originator of the uttered voice, and a generation process.
A voice dialogue method characterized by performing. To the processor
The acquisition process to acquire the reliability of each word that composes the word string related to the spoken voice, and
Based on the reliability acquired by the acquisition process, a generation process for generating a back sentence to be heard back to the originator of the uttered voice, and a generation process.
A voice dialogue program characterized by executing.

Images