JP2003141113A - Translating device, voice translating method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To realize precise and high-speed translation processing by presenting an example candidate from a speech recognition result. SOLUTION: The input voice is speech-recognized, and a word having the highest one of previously provided words is extracted as a candidate. An example is prepared for each scene, and related words are related to the respective examples. The extracted word and the word of each example are compared, scoring is performed according to the conformity, and the words having a high score are presented as candidates. A suitable word is selected from the presented candidates, and parallel translation to the selected example is output.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a translation device and the like capable of outputting an accurate parallel translation at high speed.

[0002]

2. Description of the Related Art Conventionally, when a conversation is made with a local person during an overseas trip or the like, a portable conversation translation device for supporting the conversation has been used. The speech translation in this type of conversation translator recognizes a natural language speech sentence uttered by a speaker by an inference method, converts the recognition result into a character string, and selects a corresponding character string in another language. . However, it is almost impossible to completely cover various language phenomena that appear in the process of communication between human beings,
For this reason, erroneous processing may be performed when an uncovered language phenomenon is spoken.

Therefore, recently, in order to solve such an adverse effect, a large number of sets of examples of original sentences and their bilingual examples are stored in an example database, and when the original sentence is input by voice in a predetermined language, The voice sentence is voice-recognized, the word that is the keyword is extracted from the voice of the conversation sentence of the recognition result, an example including the extracted word is searched from many examples in the example database, and the example of the search result is extracted. Translators that select an example with the highest degree of word matching and translate the original sentence according to the bilingual example of the example have also appeared.

[0004]

With the conventional method of using an example, it is possible to convey one's will to a person who cannot understand the other person's words, but a word of the other person's response to the one's will. Was not considered. For this reason, in some cases, the inquiry from the other party could not be understood or could not be heard, and communication with the other party could not be carried out smoothly.

The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a translation device, a speech translation method, and a program that can output translated sentence examples more accurately.

[0006]

In order to achieve the above object, a translation device according to a first aspect of the present invention is a translation device for translating between a pair of other languages among a plurality of natural languages. Of a plurality of languages, a language setting unit that sets one of the pair of other languages, a voice recognition unit that acquires a voice sentence in the language set by the language setting unit, and recognizes the voice, and the voice recognition unit. A word recognition means for recognizing a word included in the acquired voice sentence based on a voice recognition result by, a plurality of sentence examples for each language, at least one word associated with each sentence example, and a parallel translation sentence example of each sentence example. , A sentence example in which a plurality of corresponding sentence examples are selected based on the degree of coincidence between the sentence example storage unit that stores, and the word recognized by the word recognition unit and the word stored in the sentence example storage unit. Weather Of the sentence examples selected by the selecting means and the sentence example candidate selecting means, the sentence example candidate outputting means for outputting to the user in a selectable manner, and the sentence example selected by the user among the sentence examples output by the sentence example candidate outputting means, The corresponding bilingual sentence example is acquired from the sentence example storage unit and output, and the target language for voice recognition by the voice recognition unit is changed from the language set by the language setting unit to the other language. And a target language switching unit for switching.

According to this structure, the voice sentence of the first language (for example, Japanese) that has been input by voice is recognized by voice recognition, and the words are extracted and compared with the words associated with the prepared sentence examples. To do. As a result of the comparison, a predetermined number of sentence examples in which the degree of word matching is high are selected, and the speaker is made to select the optimum one. Then, the parallel translation (second language: English, for example) of the selected sentence example can be presented to the other party by being output to an output device such as a display device or a voice synthesizer. Therefore, it is possible to accurately output the parallel translation of the content that the speaker wants to convey. Also, for example, after outputting the parallel translation, the target language can be switched to the other,
It is possible to deal with the translation of the response by the respondent, and the translation processing can be performed without hindering the progress of the conversation.

In the above translation apparatus, it is preferable that the sentence example storage means classify and store sentence examples and bilingual sentence examples for each scene. In this case, the voice recognition means recognizes the voice sentence as a voice recognition target. The sentence example selection unit may include a scene designation unit that designates a scene to be used in advance, and the sentence example candidate selection unit may select a candidate from a sentence example corresponding to the scene designated by the scene designation unit.

[0009] In this case, the response sentence example storage means stores in advance the response sentence example for the parallel translation example output by the parallel translation example output means for each scene, and the scene is designated by the scene designation means. It is desirable to further include a response sentence example acquisition unit that obtains a response sentence example corresponding to the response sentence example storage unit, in this case, the sentence example candidate selection unit, after switching the target language by the target language switching unit, Sentence example candidates may be selected from the response sentence examples acquired by the response sentence example acquisition means.

According to the above configuration, sentence examples are classified in advance for each scene, and the speaker is allowed to select the scene in advance. Since candidate sentence examples are selected from the sentence examples corresponding to the selected scene, the sentence examples intended by the speaker can be selected more accurately.

Further, since the response of the other party to the presented parallel translation is predicted corresponding to the selection of the scene and is stored in the memory or the like in advance, the processing when the other party selects the response sentence example can be performed at high speed. it can. In this case, since the response sentence examples are also classified for each scene, the sentence examples intended by the responder can be accurately selected.

In the above translation apparatus, the speech recognition means includes, for the plurality of languages, a phoneme model in which each phoneme included in the speech is modeled and a language model in which phoneme pattern series information of a plurality of types of words is registered. It is desirable to include a model storage unit that is stored in advance. In this case, the input voice sentence is recognized by referring to the phoneme model and the language model stored in the model storage unit. The word recognizing means arranges a group indicating a silent state and a group including a word that is not registered in the language model before and after a group including a word to be recognized. A connection rule storage means for pre-storing information indicating a rule, and based on the inter-group connection rule stored in the connection rule storage means, From a speech statement whose serial speech recognition means recognizes speech, obtains the maximum likelihood for each group, based on the maximum likelihood calculated said, it can be made to extract a word to be recognized.

According to such a configuration, for example, silent parts before and after utterance of a spoken voice sentence and words that have no meaning in themselves, such as particles, are distinguished, and only necessary words are extracted. Therefore, the sentence example candidates can be selected more accurately.

The translation apparatus may further include position information acquisition means for acquiring position information of the current position, in which case the language setting means is based on the current position information acquired by the position information acquisition means. Then, one of the pair of other languages can be set.

According to such a configuration, for example, by providing the translation device with a receiving device compatible with GPS (Global Positioning System) or the like, the current position information of the translation device is acquired and Set the target language based on. That is, for example, the language used by the user is set by default, and the language used by the other party is determined by the current position and automatically set. As a result, for example, even when the language used in different regions varies within the same country, the user can set an appropriate language without being aware of it.

In the translation device, the speech recognition unit discriminates the language that composes the speech sentence by recognizing the acquired speech sentence by referring to all of the phoneme model and the language model. The language setting means may include a spoken language determining means, in which case the language setting means outputs a message in a plurality of languages that prompts a predetermined language to be spoken in a native language, and the message determined by the spoken language determining means. It is possible to set the language of the predetermined wording spoken in accordance with.

According to this structure, for example, when the translation device is started, a message prompting the user to speak a predetermined word (eg, greeting) in his native language is displayed on the display device in a plurality of languages. Since the spoken language is determined by recognizing the spoken word in response to this message, the set language can be appropriately changed for each use.

The translation device is preferably composed of a portable terminal device.

With such a configuration, the translation apparatus according to the present invention can be carried, for example, when traveling abroad, and the translation according to the present invention is required in a situation where communication in another language is required. The device can be utilized.

In the above translation device, the target language switching means may switch the language of the voice recognition target to the other language when the parallel translation example output means outputs the parallel translation example.

With such a configuration, it is possible to automatically switch the target language to the other party's language each time the bilingual translation is output.

Further, when the translation device is composed of a portable terminal device, it may further comprise a speaker discrimination means for discriminating which is the speaker. In this case, the target language switching means is The target language can be switched based on the discrimination result of the speaker discrimination means.

Further, in this case, the speaker discrimination means is
A voice collecting unit that collects a voice of a speaker and supplies a voice signal to the voice recognition unit, the voice collecting unit including at least two voice collecting units (for example, directional microphones) capable of specifying a voice input direction. It is possible to determine which of the speakers is based on the difference in the input direction identified by each of the voice collecting units, and the target language switching unit is the speaker identified by the speaker determining unit. The target language can be switched based on the utterance timing of.

Alternatively, device direction specifying means for specifying the direction of the translation device when the sentence example candidate output means outputs the sentence example candidate, and the direction of the translation device when the parallel translation sentence example output means outputs the parallel translation sentence example, The apparatus may further include a direction change detecting unit that detects a change in the direction of the translation device specified by the device direction specifying unit, and in this case, the target language switching unit may change the detection result of the direction change detecting unit. Based on this, the target language can be switched.

Further, in this case, the translation apparatus further includes display means, the sentence example candidate output means, and display control means for displaying the output from the parallel translation sentence example output means on the display means. Can be
In this case, when the target language switching unit switches the target language based on the determination result of the output target determining unit, the display control unit may change the display direction according to the determined output target. Good.

According to the above configuration, in the case where a conversation proceeds while handing over the translator between the two parties, the target language is automatically determined based on which speaker is the speaker. You can switch. Here, it is assumed that the voice collecting device (for example, a microphone) for inputting voice into the voice recognition unit of the translation device has directivity,
If at least one such voice collecting device is provided on each of the front surface and the back surface of the translation device, when face-to-face conversations are made, it is possible to use the voice input direction of the two persons. You can determine which of them is speaking. Therefore, the target language can be automatically switched by any person who inputs a voice for voice recognition.

Alternatively, the target language can be automatically switched depending on the content of the output information and the direction of the device.
That is, when the person who refers to the sentence example candidate is one of the two persons, the person who refers to the parallel translation can be regarded as the other person. Therefore, when the directions of the translation device are specified at the time of outputting the sentence example candidates and at the time of outputting the bilingual sentence examples, the passing operation can be detected by detecting the change in the device direction. If the target language is automatically switched in accordance with this passing operation, the translation process can be sequentially performed as the conversation progresses. Furthermore, in this case, the display screen is turned upside down, for example, depending on which person the information is presented to (whether the output information is a sentence example candidate or a bilingual sentence example). ,
There is no need to turn around.

In order to achieve the above object, a speech translation method according to a second aspect of the present invention comprises a language designating step of designating one of a pair of other languages according to the utterance situation of a speaker. A voice recognition step of recognizing a voice by acquiring a voice sentence in the language designated in the language designation step;
A word recognition step of recognizing a word included in the acquired voice sentence based on the voice recognition result in the voice recognition step, a plurality of sentence examples for each language, and at least one word related to each sentence example, and Based on the degree of coincidence between the sentence example accumulating step of preliminarily accumulating the parallel translation example of the sentence example and the word recognized in the word recognizing step, and the word accumulated in the sentence example accumulating step, and corresponding. A sentence example candidate selection step of selecting a plurality of sentence examples, a sentence example candidate output step of selectively outputting the sentence example selected in the sentence example candidate selection step, and a sentence example output in the sentence example candidate output step are selected. A parallel translation sentence example output step of acquiring and outputting a parallel translation sentence example corresponding to the sentence example.

In order to achieve the above-mentioned object, a program according to a third aspect of the present invention is configured such that a computer specifies a language for specifying a pair of other languages out of a plurality of natural languages, a voice sentence acquiring section for acquiring a voice sentence. A voice recognition unit that recognizes a voice sentence acquired by a designation unit and the voice sentence acquisition unit based on one language designated by the language designation unit; and the voice sentence based on the voice recognition result by the voice recognition unit. A word recognition unit for recognizing a word included in, a sentence example storage unit that stores a plurality of sentence examples for each language, at least one word related to each sentence example, and a parallel translation sentence example of each sentence example in association with each other; Based on the degree of coincidence between the word recognized by the recognition unit and the word accumulated in the sentence example accumulating unit, a sentence example candidate selecting unit that selects a plurality of corresponding sentence examples, and a plurality of sentence examples selected by the sentence example candidate selecting unit. Election Phrases candidate output unit for outputting capable,
Among the sentence examples output by the sentence example candidate output unit, the bilingual sentence example output unit that outputs the bilingual sentence example corresponding to the selected sentence example, and the voice recognition unit performs voice recognition upon the output of the bilingual sentence example by the bilingual sentence example output unit. It is characterized in that it functions as a target language switching unit that can switch the target language to the other.

[0030]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described below with reference to the accompanying drawings.

FIG. 1 is a diagram showing the external appearance of a translation apparatus 100 according to the embodiment of the present invention. As illustrated, the translation device 100 is a portable device, and includes a voice input unit 110, an input unit 200, and an output unit 300 as its external configuration.

The voice input unit 110 is composed of, for example, a microphone, and is for inputting a voice sentence for performing voice recognition described later.

The input unit 200 is composed of, for example, various button groups, and is for performing various selection operations described later.

The output unit 300 is composed of, for example, a liquid crystal display device, and displays candidate sentence examples, parallel translation sentence examples, and the like, which will be described later.

Next, the internal structure of the translation apparatus 100 will be described with reference to FIG. FIG. 2 is a block diagram schematically showing the internal configuration of the translation device 100.

As shown, the translation device 100 is shown in FIG.
In addition to the configuration shown in, the voice recognition unit 120 and the grammar file 1
30, a model storage unit 140, a likelihood calculation unit 150, a word output unit 160, an example management unit 170, an example storage unit 180, and a work area 190.

Further, more specifically, the input section 200 includes a language designating section 210, a scene designating section 220, and an example designating section 230. The language designating section 210 designates a target language. The scene designating section designates a scene in which the phrase to be translated is used, based on a scene prepared in advance. The example specifying unit 230 is for selecting and specifying an example intended by the user from the example candidates extracted by the translation apparatus 100.

The voice recognition section 120 includes an AD conversion section 121,
The power analysis unit 122, the voice section detection unit 123, and the feature extraction unit 124 are provided.

The AD converter 121 A / D converts the voice (analog signal) input to the voice input unit 110, and converts this voice into a digital voice signal (for example, a PCM signal) shown in time series. .

The power analysis section 122 includes an AD conversion section 121.
A / D-converted digital audio signal is divided into a plurality of frames at predetermined time intervals (2.0 to 4.0 milliseconds) by a Hamming window such as a time window, and each divided frame is divided. The audio data is cut out from and the power component is obtained.

The voice section detector 123 is a power analyzer 1.
A frame in which the power component obtained by 22 exceeds a predetermined threshold is detected as a voice section. For example, a frame in which the power component falls below the threshold within 1 second is not set as a voice section.

The feature extraction unit 124 includes a voice section detection unit 12
For the voice section detected by 3, the acoustic feature vector X (t) is calculated based on the power components calculated by the power analysis unit 122.

The grammar file 130 is a file that defines the grammatical rules for each language supported by the translation device 100, and is referred to in the word recognition process described later. In the present embodiment, for ease of description, the number of languages that translation apparatus 100 can handle is two, and each is described as a “first language” and a “second language”. In the present embodiment, the first language is Japanese and the second language is
An example when the language is English will be described.

The model storage unit 140 includes a first language acoustic model storage unit 141 and a first language language model storage unit 14.
2, a second language acoustic model storage unit 143 and a second language language model storage unit 144. The acoustic model storage units 141 and 143 store, for each language, a phoneme model obtained by modeling all phonemes forming the speech to be recognized. For example, a hidden Markov model (HMM) is applied as the phoneme model. In addition, the language model storage units 142 and 144 store a word dictionary in which phoneme pattern sequence information for each word is registered for each language.

The likelihood calculator 150 calculates the acoustic feature vector X
The continuous phoneme recognition is performed for each frame with reference to (t) and the phoneme model based on the HMM, and the one having the maximum total likelihood is extracted as a first-rank candidate word.

The word output section 160 outputs the word extracted by the likelihood calculation section 150.

The example management section 170 uses the scene name register 17
1, an example group name register 172, and a predicted response example management table 173. The scene name register 171 holds the scene name designated by the scene designation unit 220. The example group name register 172 holds an example group file name that is a target in an example candidate extraction process described later. The predicted response example management table 173 is a table for designating an example indicating a response to the example based on the example candidates selected for each language. That is, as shown in FIG. 3, the example group file in the first language and the response example group file name in the second language are recorded in association with each other. It is also assumed that a similar table is prepared in which the example group files in the second language and the response example group file names in the first language are associated with each other.

The example storage unit 180 includes an example database (first language) 181 and a prediction response example database (second language).
Language) 182, example database (second language) 183,
Prediction response example database (first language) 184
Equipped with.

The example databases 181 and 183 are
For each language, example data is stored in a hierarchical directory structure as shown in FIG. This hierarchical structure is constructed based on the scene that needs translation. In the present embodiment, it is assumed that the first layer to the third layer are scene setting layers, and the example group file is stored in the fourth layer, which is the lowest layer.

That is, in the example of FIG. 4, a plurality of example group files corresponding to “emergency” → “at the hospital” → “symptom” are prepared in the fourth hierarchy. Here, for each symptom (for example,
"Headache", "abdominal pain", etc.)
It is assumed that 1 ”,“ hospJ02 ”, and“ hospJ03 ”are prepared.

FIG. 5 shows an example of the data structure of the example group file "hospJ01". In this example, example file 1
Is a file that stores examples of "headache". As shown, the example group file 1 contains "headache".
There are n types of examples related to. Since this "hospJ01" is the lowest layer in the hierarchy of "emergency" → "at the hospital" → "symptoms", there are prepared examples for complaining headaches at hospitals such as "I have a headache." . And
Each example is associated with a keyword for deriving the example. In the present embodiment, five keywords, word 1 to word 5, are set as keywords, and the larger the numerical value, the higher the specificity. Furthermore, the parallel translation example sentence in the second language (English) of each example is associated.

A scoring table as shown in FIG. 6 is prepared for each example group file. This scoring table is referred to in the score calculation performed in the example candidate extraction process to be described later, and shows the points assigned to each of the above keywords (word 1 to word 5). As described above, the higher the word number, the more specific the word is. Therefore, the score is also set higher as the word number increases. The distribution of points may be different for each example group file.

Predictive Response Example Databases 182 and 1
Reference numeral 84 stores an example in which the other party responds to the finally presented example. That is, the examples used by the speaker are stored in the example database, and the examples used by the responder are stored in the predicted response example database. Therefore, the second language example database 181 contains the second
The language prediction response example database 182 corresponds to the second database.
The language example database 183 is configured to correspond to the predicted response example database 184 of the first language.

The work area 190 is an example area (first
Language) 191, a response example area (second language) 192, an example area (second language) 193, and a response example area (first language). Each area has an example storage 18
The example file stored in 0 is expanded and each processing described later is performed.

Next, the translation device 10 according to the present embodiment
The operation of 0 will be described with reference to the drawings.

FIG. 7 is a flow chart for explaining the operation of translation apparatus 100. First, a screen for selecting the first language or the second language is displayed on the output unit 300 of the translation device 100, and one is selected by operating the input unit 200 (step S101). That is, according to the operation of the input unit 200, the language designation unit 210 sets any language. Here, a case where the first language (Japanese) is set will be described as an example.

Next, a scene menu for designating a scene is displayed on the output unit 300 as shown in FIG. 8 (step S102). The user operates the input unit 200 to select a desired scene. As described above, since the scenes have a hierarchical structure, a predetermined scene (hereinafter referred to as “detail scene”) is selected until the scene prepared in the third hierarchy, which is the lowest layer of scene specification, is selected. The menu screen is displayed.

When the detailed scene is selected (step S10)
3: Yes), the scene designation unit 220 stores the scene name of the selected scene in the scene name register 171 of the example management unit 170 (step S104). Below, detailed scene "symptoms"
The case where is selected will be described as an example.

The translation device 100 uses the scene name register 171.
Example group files (that is, "hospJ01", "hospJ02", "hospJ0" corresponding to the scene name "symptom" stored in
3 ”) is acquired from the example database (first language) 181 and stored in the example area (first language) 191 (step S105).

Next, the translation apparatus 100 has the output unit 300.
A message prompting voice input to the voice input unit 110 is displayed on. As a result, when the user inputs a voice (step S106: Yes), the voice recognition process is started (step S200). If no voice is input for a predetermined time or more (step S106: No, step S107: Yes), the process ends.

Next, the voice recognition process of step S200 will be described with reference to the flowchart of FIG. Here, a case will be described as an example in which the voice sentence “Head is very painful” is input to the voice input unit 110.

First, the AD converter 121 A / D-converts the input voice "Head is very painful", and the power analyzer 122 converts the converted digital voice signal into a plurality of frames at predetermined time intervals. Then, the power components of each frame are calculated (step S201).

Next, the voice section detector 123 detects the voice section based on the power analysis in step S201 (step S202). The power component of each frame calculated in step S201 is compared with a predetermined threshold value, and a frame exceeding the threshold value is extracted as a voice segment.

In this process, it is determined whether or not the necessary voice sections for extracting the candidate words are detected (step S203). If the required number of voice sections has not been detected (step S203: No), the process ends. When the necessary number of voice sections is detected, the feature extraction unit 124 determines the acoustic feature vector X (t) of the voice section extracted in step S203 based on the power component of each frame calculated in step S201. Calculate (step S204).

Next, the likelihood calculating section 150 determines in step S2.
According to the speech feature vector X (t) calculated in 04, the hidden Markov model (HMM) as a phoneme model stored in the first language acoustic model storage unit 141, and the connection rule stored in the grammar file 130. , Cumulative likelihood calculation is performed (step S205).

Regarding the connection rule of this HMM model, FIG.
This will be described with reference to 0. This rule is used when recognizing a word included in a voice sentence by using the word spotting method.
It defines the connection relationship of the MM model, is stored in advance in the grammar file 130, has the concept of a group, and each group includes one or more words. Its structure is to put silence and a garbage model (model corresponding to all phonemes) before and after a plurality of words (groups) that are the recognition target words, and absorb the unrelated utterances before and after. To do.

Here, the maximum likelihood value is calculated according to the following conditions (step S300). The likelihood calculation from the first frame is performed only on the words included in the start group (0, 1, 3 group) defined in the connection rule. -The 0th group includes only the word B (here, both B and E are words). Further, the 0th group may be changed to the 0th group, the 1st group, and the 3rd group, and the final state of B is the 0th group,
All the words included in the first group and the third group have transited to the leading states, and in the normal Viterbi algorithm, the transition having a large cumulative likelihood is prioritized, and the cumulative likelihood calculation is advanced every time.

This processing will be described with reference to the flowchart shown in FIG. That is, the transition destination group is determined for each group (step S301), and the word indicating the maximum likelihood is determined (step S302). Then, the maximum likelihood value of the word determined in step S302 is acquired (step S303).

At this time, as data necessary for the back trace of the post-processing, of all the words in each group for each frame, which word has the highest likelihood in the final state, and the likelihood of that word, The value and the value of the start frame of the word are obtained and stored (step S206).

When the cumulative likelihood calculation of all the frames is completed (step S207: Yes), the recognition result is obtained by tracing back the stored likelihood calculation result (backtrace) (step S208).

Among the likelihoods of all words included in each of the final groups 1, 2, and 4, the one with the highest cumulative likelihood is obtained. For example, in the case of the second group (E), E becomes an output and traces back to the start frame of E,
At that time, 1, 2, which are also connected to the second group
The maximum likelihood of all the words included in the group of 4 is obtained.

In this way, when the start frame is traced back to 0 frame (step S209: Yes), the route showing the maximum likelihood is output.

From this recognition result, a maximum of three keywords are detected per utterance (step S210).

In the present embodiment, since the input voice sentence is "My head hurts a lot," as shown in FIG. 12, as the recognition result, "BBB head G very G want GGEE".
E ”is acquired. Here, B indicates silence before pronunciation,
G indicates a garbage model that absorbs utterances other than keywords, and E indicates silence after utterances, so the extracted keywords are "Atama,""very," and "taitai."

Next, based on the result of the voice recognition process of step S200, the example candidate extraction process (step S40)
0) is executed. This example candidate extraction process will be described with reference to the flowchart in FIG.

Likelihood calculation section 150 uses the words extracted by the above-mentioned speech recognition process, that is, "Atama", "very",
“I want” is output to the word output unit 160 (step S401).

Next, the head file name of the target example group file is set in the example group name register (step S
402). In the above example, “hospJ01” indicating the first file of the example group files “hospJ01”, “hospJ01”, and “hospJ03” corresponding to the “symptom” is set.

Therefore, the following processing is performed using the example group file stored in the example area (first language) 191 in step S105 and corresponding to "hospJ01".

For each example of "hospJ01", the corresponding keyword and the word acquired in step S401 are individually compared to determine whether there is a match (steps S403, 404). If there is a match here,
Score calculation is performed based on the points assigned in the scoring table shown in FIG. 5 (step S405).

This score calculation processing will be described with reference to FIG. First, with the words "Atama,""very," and "I want to" acquired in step S401, the example No. When the keyword 1 is compared, "Atama" and "Itai" match. Here, the example No. In 1, "Atama" is set to word 1 and "Itai" is set to word 4, so based on the scoring point table, "Atama" has 80 points and "Itai" has 95 points, for a total of 175 points. Is attached.

Similarly, score calculation is performed for all examples of the "hospJ01" file (step S40).
6: No, S407). When the score calculation is completed for all the examples (step S406: Yes), the same process is performed for the next example group file (step S4).
08: No, S409).

When the above process is completed for all the example group files (step S408: Yes) and no word match is detected (step S410: No).
Returns to step S111 (FIG. 7) and returns to the output unit 300.
A voice re-input request message is displayed on.

On the other hand, when there is a word match and a score is assigned (step S410: Yes), a predetermined number of examples are acquired as candidates from the top of the assigned score (step S411), and the corresponding parallel translation example is used. It is also stored in the example area (first language) 191 and the process returns to the flow shown in FIG. In this embodiment, the top three examples are acquired.

The candidate examples acquired in step S411 are displayed on the output unit 300 as shown in FIG. 15 (step S108). Here, the three candidate examples are displayed so that they can be selected. That is, by operating the input unit 200, it is possible to select an example closest to the sentence spoken at the time of voice input.

Here, when the predetermined cancel operation is performed without selecting the example (step S109: No, S11).
0: Yes) displays a predetermined voice re-input request message on the output unit 300 (step S111).

On the other hand, when the example is selected (step S
109: Yes), the parallel translation example corresponding to the selected example is acquired from the example area (first language) 191, and is displayed on the output unit 300 as shown in FIG. 16 (step S112).

When the parallel translation is displayed, the example group file update process (step S500) is executed. This example group file update process will be described with reference to the flowchart shown in FIG.

First, the example group name register 172 stores the example group file name corresponding to the example selected in step S109.
(Step S501).

Next, referring to the predicted response example management table 173, the example group file name (predicted response example group name) for the second language corresponding to the example group file name obtained in step S501 is obtained (step). S502).

The predicted response example group name acquired in step S502 is set in the example group name register 172 (step S).
503).

All example group files corresponding to the predicted response example group names set in step S503 are acquired from the predicted response example database (second language) 182 and stored in the response example area (second language) 192. The process ends (steps S504 and S505).

After this, the parallel translation displayed in step S112 is presented or spoken to the other party. As shown in FIG.
The output unit 300 is displayed to speak a response to the displayed parallel translation. Therefore, the responder inputs, into the voice input unit 110, a voice sentence responding to the displayed “I have a terrible headache.” (The head is very painful).

After that, the same processing (speech recognition processing, example candidate extraction processing, example selection, parallel translation display of selected examples) as the above-described processing is performed for the second language (English). In step S504, since the predicted response example is already stored in the response example area (second language) 192, the response example candidate is selected from among them. That is, as shown in FIG. 18A, the three second
Language response example candidates are displayed on the output unit 300 in a selectable manner. Then, the responder operates the input unit 200 to select the most suitable example. A parallel translation (first language: Japanese) of the selected response example is displayed on the output unit 300 as shown in FIG.

As described above, according to the translation apparatus 100 according to the embodiment of the present invention, the user is allowed to select an example candidate close to the uttered content, the bilingual translation of the selected example is displayed, and Since the response to the example is predicted in advance and the voice is recognized, the translation process can be performed accurately and at high speed.

In the above-described embodiment, the output unit 300 is composed of, for example, a display device to display the parallel translation of the selected example. On the other hand, the output unit 300 may be connected to some electronic circuit and the electronic circuit may be operated according to the output result from the output unit 300. For example, by connecting a voice synthesizer to the output unit 300, the parallel translation may be voice output.

The translation device 100 has been described as a portable device, but in this case, the translation device 1
00 does not have to be a dedicated device. For example, general-purpose PDA
(Personal Digital Assistance)
By installing a program for realizing each of the above processes, translation apparatus 100 according to the present embodiment
May be realized. In addition, mobile phones and PHS (Personal
A similar program may be installed in a mobile communication terminal such as a Handyphone System) terminal to cause it to function as the translation device 100.

In the above embodiment, the corresponding languages are the first language (Japanese) and the second language (English) for ease of explanation.
However, the language used by the speakers may be selected from many languages. In this case, the model storage unit 140, the example storage unit 180, the work area 190, and the like can be appropriately expanded.

Alternatively, by providing the translation device 100 with a communication function, these storage areas may be prepared in, for example, an external host computer and the like, and the processing may be performed via the Internet or the like. With such a configuration, even a portable device having a limited storage capacity can support more languages.

In the above embodiment, the language selection is manually set, but the language selection method is arbitrary. For example, the language may be automatically switched when the parallel translation is displayed.

Alternatively, the language may be switched by discriminating which is the speaker. in this case,
For example, the voice input unit 110 is configured by a directional microphone or the like, and further, the translation device 100 is configured to include at least two voice input units. Then, when the voice is acquired by the plurality of microphones, the direction of the speaker is detected from the difference in the arrival time. With such a configuration, it is possible to switch the language upon the detection of voices from different directions after the end of one utterance.

Alternatively, the language may be switched by determining the output target of the information output by the translation apparatus 100. That is, it is considered that the speaker is looking at the screen when the candidate sentence example is displayed, and the screen is presented to the other party when the bilingual sentence example is displayed. Therefore, for example, it is configured to determine the direction of the translation device 100 when the candidate sentence example is displayed and when the bilingual sentence example is displayed, and the target language is switched based on the change in this direction. You may In this case, the display direction on the output unit 300 may be vertically inverted, for example, depending on the direction of the translation device 100. Thereby, when handing over the translation device 100 between two parties, it is not necessary to invert the translation device 100 itself.

In addition, the translation device 100 has a GPS (Global P
ositioning System: A global positioning system may be provided to automatically set the language used according to the current position. In this case, the language used by the owner of the translation apparatus 100 is set by default, and the language of the other party is automatically set as appropriate. According to such a configuration, for example, even when the language used in each region is different in the same country, an appropriate language can be set without the user's awareness.

As described above, the language used by the owner may be fixed by default, or may be set by using voice recognition. For example, when the translation apparatus 100 is activated, a message prompting the user to speak a predetermined language (eg, greetings) in his native language is displayed on the output unit 300 (display device) in multiple languages. The language is identified by the voice recognition unit 120 voice-recognizing the word spoken according to this message. In this case, the model storage unit 1
Language discrimination is performed by referring to acoustic models and language models for all languages stored in 40. Then, the determined language can be set as the target language.

The translation device 100 described above is not limited to a portable device, and can be realized by using a general-purpose computer such as a personal computer as a platform. For example, the voice input unit 110 and the output unit 300 are each a microphone connected to a general-purpose computer,
It is realized by a display device. The grammar file 130, the model storage unit 140, and the work area 190 are realized by allocating areas corresponding to the grammar file 130, the model storage unit 140, and the work area 190 in a memory. The model stored in the model storage unit 140 is read in advance from an external device and stored in the memory before performing the voice recognition process. Power analysis unit 122, voice section detection unit 123, feature extraction unit 124, and likelihood calculation unit 1
50 is implemented by a CPU (Central Processing Unit) executing a program stored in a memory.

With such a configuration, the speech translation method according to the present invention can be applied, for example, when communicating with a person in another language via the Internet or the like.

[0106]

As described above, according to the present invention,
An accurate parallel translation can be output at high speed.

[Brief description of drawings]

FIG. 1 is a diagram showing an external appearance of a translation device according to an embodiment of the present invention.

FIG. 2 is a block diagram showing a configuration of the translation device of FIG.

FIG. 3 is a diagram showing an example of information recorded in a predicted response example management table shown in FIG.

FIG. 4 is a diagram for explaining the structure of information recorded in the example database shown in FIG.

5 is a diagram showing an example of an example group file recorded in the example database shown in FIG.

6 is a diagram showing an example of a scoring table recorded in an example database together with the example group file shown in FIG.

FIG. 7 is a flowchart for explaining an operation of the translation device according to the embodiment of the present invention.

8 is a diagram showing a display example of a scene setting screen displayed on the translation device in the processing shown in FIG. 7. FIG.

9 is a flowchart for explaining a voice recognition process in the process shown in FIG.

10 is an HMM used in the speech recognition process shown in FIG.
It is a figure for demonstrating the connection rule of a model.

FIG. 11 is a flowchart for explaining maximum likelihood calculation processing in the voice recognition processing shown in FIG. 9.

FIG. 12 is a diagram for explaining a mechanism of outputting a recognized word in the voice recognition process shown in FIG. 9.

FIG. 13 is a flowchart illustrating an example candidate extraction process in the process shown in FIG. 7.

FIG. 14 is a diagram for explaining a score calculation process in the example candidate extraction process shown in FIG.

15 is a diagram showing a display example of an example candidate display screen displayed by the processing shown in FIG.

16 is a diagram showing a display example of a parallel translation display screen displayed by the processing shown in FIG.

FIG. 17 is a flowchart illustrating an example group file update process in the process shown in FIG. 7.

FIG. 18 is a diagram showing a display example of a screen displayed by an operation of a responder in the translation device according to the embodiment of the present invention, (a) showing a display example of an example candidate display screen,
(B) shows a display example of the parallel translation display screen.

[Explanation of symbols]

100 ... Translation device, 110 ... Voice input unit, 120 ... Voice recognition unit, 140 ... Model storage unit, 150 ... Likelihood calculation unit,
170 ... Example management unit, 180 ... Example storage unit, 200 ... Input unit, 300 ... Output unit

─────────────────────────────────────────────────── ─── Continuation of front page (51) Int.Cl. ⁷ Identification code FI theme code (reference) G10L 15/22 G10L 3/00 571U 571V F term (reference) 5B091 AA03 BA19 CB12 CB24 CC15 DA11 5D015 AA05 AA06 BB01 BB02 HH00 KK02

Claims (14)

[Claims] 1. A translation device for translating between a pair of other languages among a plurality of natural languages, a language setting means for setting one of the pair of other languages among the plurality of languages, and the language. A voice recognition unit that acquires a voice sentence in the language set by the setting unit and recognizes the voice, and a word recognition unit that recognizes a word included in the obtained voice sentence based on the voice recognition result by the voice recognition unit, For each language, a plurality of sentence examples, at least one word associated with each sentence example, and a parallel example sentence example of each sentence example, a sentence example storage unit that stores the corresponding sentence example, a word recognized by the word recognition unit , A sentence example candidate selecting means for selecting a plurality of corresponding sentence examples based on the degree of coincidence with the words stored in the sentence example storing means, and a sentence example selected by the sentence example candidate selecting means, to be selectably output to the user. A sentence example candidate output means, among the sentence examples output by the sentence example candidate output means, a bilingual sentence example corresponding to the sentence example selected by the user, parallel translation sentence example output means for acquiring and outputting from the sentence example storage means, The language that is the target of voice recognition by the voice recognition means,
A target language switching unit for switching from the language set by the language setting unit to the other language, the translation device. 2. The sentence example storage means classifies and stores sentence examples and bilingual sentence examples for each scene, and the voice recognition means includes a scene designation means for previously designating a scene in which a voice sentence as a voice recognition target is used. The translation apparatus according to claim 1, wherein the sentence example candidate selection means selects candidates from the sentence examples corresponding to the scene designated by the scene designation means. 3. A response sentence example storage unit that stores in advance a response sentence example for the parallel translation example output by the parallel translation example output unit for each scene, and responds to the scene by the scene designation by the scene designation unit. Response sentence example acquisition means for obtaining a response sentence example from the response sentence example storage means, the sentence example candidate selection means, after the switching of the target language by the target language switching means, the response acquired by the response sentence example acquisition means The translation device according to claim 2, wherein sentence example candidates are selected from the sentence examples. 4. The speech recognition means stores in advance, for each of the plurality of languages, a phoneme model in which each phoneme included in the speech is modeled and a language model in which phoneme pattern sequence information of a plurality of types of words is registered. A model storage unit is provided, and recognizes an input speech sentence by referring to the phoneme model and the language model stored in the model storage unit, and the word recognition unit becomes a recognition target. A connection that stores in advance information indicating an inter-group connection rule for defining a connection relationship in which a group indicating a silent state and a group including a word not registered in the language model are arranged before and after the group including a word. A rule storage unit is provided, and based on the inter-group connection rule stored in the connection rule storage unit, each group is selected from the voice sentences recognized by the voice recognition unit. Obtains the maximum likelihood per flop, based on the maximum likelihood calculated said, extracts words to be recognized, the translation device according to any one of claims 1 to 3, characterized in that. 5. The apparatus further comprises position information acquisition means for acquiring position information of a current position, wherein the language setting means is based on the current position information acquired by the position information acquisition means, and the language setting means acquires the position information. The translation apparatus according to any one of claims 1 to 4, wherein one of them is set. 6. The speech recognizing means for recognizing the acquired speech sentence by referring to all of the phoneme model and the language model, thereby deciding the language constituting the speech sentence. The language setting means outputs a message in a plurality of languages that prompts the user to speak a predetermined language in a native language, and the predetermined language spoken according to the message determined by the spoken language determination means. The translation device according to any one of claims 1 to 5, wherein the language of the wording is set. 7. The translation device according to claim 1, wherein the translation device comprises a portable terminal device. 8. The target language switching means switches the speech recognition target language to the other language upon the output of the bilingual sentence example by the bilingual sentence example outputting means, according to any one of claims 1 to 7. The translation device according to item. 9. A speaker discrimination means for discriminating which is a speaker is further provided, and the target language switching means switches the target language based on a discrimination result of the speaker discrimination means. The translation device according to claim 7. 10. The speaker discrimination means is a voice collection means for collecting a voice of a speaker and supplying a voice signal to the voice recognition unit, and at least a voice collection means capable of specifying a voice input direction. Two are provided, and it is determined which of the speakers is based on the difference in the input direction specified by each of the voice collecting units, and the target language switching unit is the speaker of the speaker determined by the speaker determining unit. The translation device according to claim 9, wherein the target language is switched based on the utterance timing. 11. The speaker discrimination means specifies the direction of the translation device when the sentence example candidate output means outputs the sentence example candidate, and the direction of the translation device when the parallel translation sentence example output means outputs the parallel translation sentence example. The apparatus further comprises: a device direction identification unit; and a direction change detection unit that detects a change in the direction of the translation device identified by the device direction identification unit, wherein the target language switching unit is a detection result of the direction change detection unit. The translation device according to claim 9, wherein the target language is switched based on the target language. 12. A display unit, a display control unit that causes the display unit to display the output from the sentence example candidate output unit, and the parallel translation sentence example output unit, wherein the display control unit includes the target. 10. The translation according to claim 9, wherein when the language switching unit switches the target language based on the discrimination result of the speaker discrimination unit, the display direction is changed according to the discriminated speaker. apparatus. 13. A language designation step of designating one of a pair of other languages in accordance with the utterance situation of a speaker, and a speech sentence in the language designated in the language designation step is acquired to perform speech recognition. A voice recognition step; a word recognition step of recognizing a word included in the acquired voice sentence based on a voice recognition result in the voice recognition step; a plurality of sentence examples for each language; and at least one related to each sentence example. One word, the parallel translation example of each sentence example, the sentence example accumulating step of accumulating in advance in association with each other, the word recognized in the word recognition step, and the degree of coincidence with the word accumulated in the sentence example accumulating step. Based on this, a sentence example candidate selecting step of selecting a plurality of corresponding sentence examples, and a sentence example candidate outputting step of selectively outputting the sentence examples selected in the sentence example candidate selecting step. The phrase candidate output of phrase output in step, speech translation method characterized by comprising: a bilingual phrase outputting step, the outputs to obtain the parallel translation example sentence corresponding to the selected phrase. 14. A computer comprising: a voice sentence acquisition unit for obtaining a voice sentence; a language designation unit for designating a pair of other languages among a plurality of natural languages; and a voice sentence obtained by the voice sentence acquisition unit for the language. A voice recognition unit that recognizes a voice based on one language designated by the designation unit, a word recognition unit that recognizes a word included in the voice sentence based on a voice recognition result by the voice recognition unit, and a plurality of word recognition units for each language. A sentence example storage unit that stores a sentence example, at least one word associated with each sentence example, and a parallel translation sentence example of each sentence example in association with each other, a word recognized by the word recognition unit, and a sentence example storage unit. Based on the degree of coincidence with a word, a sentence example candidate selection unit that selects a plurality of corresponding sentence examples, a sentence example candidate output unit that selectively outputs a plurality of sentence examples selected by the sentence example candidate selection unit, and the sentence example candidate output unit outputs Among the sentence examples, the bilingual sentence example output unit that outputs the bilingual sentence example corresponding to the selected sentence example, triggered by the output of the bilingual sentence example by the bilingual sentence example output unit, the language for which the voice recognition unit is the voice recognition target to the other A program that functions as a target language switching unit that enables switching.