JPH0348366A - Morpheme alanysis, syntax analysis and morpheme forming system - Google Patents

Abstract

PURPOSE:To flexibly execute composite word processing by registering the whole composite word formed through a specific character in a translating temporary dictionary to be used for translation processing such as syntax analysis as an unknown word when the composite word is not included in a dictionary. CONSTITUTION:When a composite word formed by a specific character such as a hyphen (-) forming one composite word by mutually connecting plural words is included in a test to be analyzed in a morpheme analyzing part in a translation body part (translation part) 7, the composite word is retrieved by a dictionary 8 included in the system. When the composite word is not included in the dictionary, the whole composite word is registered in the translating temporary dictionary to be used for the translation processing such as syntax analysis as an unknown word. Since the retrieval of a composite word formed by connecting plural word through a specific character such as a hyphene in the dictionary by the morpheme analyzing part is improved, proper processing interlocking with the succeeding processing of the syntax analyzing part is executed even when the syntax analysis result of the composite word succeeds or fails. Consequently, the processing of the composite word based upon the specific character such as a hyphene can be flexibly executed.

Description

DETAILED DESCRIPTION OF THE INVENTION The present invention relates to a morphological analysis method, a syntactic analysis method, and a morphological generation method in natural language analysis systems such as machine translation.

As a prior art related to the present invention, Japanese Patent Application Laid-Open No. 63-20568
There is a publication. This Japanese Patent Application Publication No. 63-20568 is
When registering in the dictionary, check for special characters such as the hyphen "ichi" that combine words into one compound word, and convert the special characters to spaces and register them in the dictionary. The present invention relates to a machine translation device that can translate even when special characters forming compound words are added.

Processing unknown words in natural language analysis is a surprisingly difficult problem. There is no such thing as a dictionary that contains all the words and expressions of this world. Inevitably, words and expressions that are not in the dictionary will appear. Since it's not in the dictionary, you have to guess what the word is and what it means.

There are several ways to do this. To put it simply, there are unknown words that start with a capital letter. This is presumed to be a proper name subscript. Since the meaning cannot be estimated, for example, if the original text is in English, the translated word will remain in the alphabet. In the case of an unknown word that starts with a lowercase letter, a specific part of speech is given if it can be estimated from the shape of the ending. If you don't have any clues, write down names, names, adjectives, and verbs that are frequently used as unknown words. I don't know the meaning of these either, so I say things like ``original J-1-r'' and ``
For example, ``original text'' + ``no''. In addition to these, there are other unknown words whose meanings can be estimated. Combinations of ``affixes'' + ``known words'' such as ``prefix'' + ``known word'' and ``known word'' + ``suffix'' and hyphens, etc. in ``known word'' - ``known word'' A compound word formed by combining special characters. These spellings as a whole are often not found in dictionaries, but the individual components are. Therefore, we construct translations for each part and estimate the entire translation. This translation structure is relatively easy when it is a combination of "affix" + "known word", but it becomes difficult when it becomes long when it is a compound word with special characters such as a hyphen. A compound word with special characters such as a hyphen is a word made up of a combination of its components, so when estimating its meaning, syntax analysis is performed between the components of the compound word with special characters such as a hyphen. It's good when you can do this well, but when you can't do it, you end up with rather incomprehensible results, which in turn has a negative impact on the parsing of the sentence to be translated that contains the compound word. Therefore, overall, it is better to analyze unknown compound words with special characters such as hyphens, and if the analysis fails, treat the whole word as an unknown word.

The present invention was made in view of the above-mentioned circumstances, and
We have improved the dictionary lookup in the morphological analysis unit for compound words that are formed by combining them with special characters such as hyphens, and in conjunction with the processing in the subsequent syntactic analysis unit, we can solve the syntax of compound words with special characters such as hyphens. We have developed a morphological analysis method, a syntactic analysis method, and a morpheme generation method that can flexibly process compound words using special characters such as hyphens by applying appropriate processing depending on whether the result is successful or not. However, this was done for the purpose of providing this information.

The present invention was made in view of the above-mentioned points ↑I11, and includes: (1) A natural speech analysis system for machine translation, which is a system for translation. , when the text to be analyzed by the morphological analysis unit contains a compound +Jf formed by special characters such as a hyphen "ichi" that combines words to form one compound word, the system Look it up in a dictionary using the prepared tablet, and if it does not exist in the dictionary, register the entire compound:6 as an unknown word in a temporary dictionary for translation to be used for translation processing such as parsing; (2) for the translation of the nail 1; Presuming that unknown word compounds registered in the temporary dictionary are nouns, adjectives, or verbs; (3) Presuming that unknown word compounds registered in the temporary dictionary for ＋'rMkitranslation1 are nouns or adjectives; (4) In a natural language analysis system such as machine translation that has at least a system dictionary for translation, the morphological analysis unit In the text to be analyzed by F+¥,
When there is a compound word formed by special characters such as a hyphen "ichi" that combines single lines to form one compound word, the compound word is looked up using a dictionary provided by the system. When the compound word does not exist in the dictionary, it is possible to look up the individual components constituting the compound word in a dictionary and use the dictionary information for translation processing such as syntactic analysis, (5) (1) and (2) above. ), (3). (4) must be combined with the special features of (6) at least a system dictionary for translation.
In natural language analysis systems such as machine translation that have
There is a compound word in the NIFt target text in the syntactic analysis section that is formed by special characters such as a hyphen "ichi" that combines single words to form a compound word. Have. ! ii' If the compound word does not exist in the book, parse the compound word using the sequence of constituent words, and if the analysis fails, proceed with the parse by treating the entire compound 17ft as an unknown word, ( 7) At least a system dictionary for translation, (natural + such as machine translation that corrects F:
Hl. In the t-analysis system, the syntactic analysis unit analyzes the text 1-, and the compound word 2, which is formed by combining il' Vtl with special characters such as the hyphen ``ichi'', which forms a compound word ``1''. When there is a compound word and the compound word does not exist in the dictionary δ held by the system, the compound word is parsed using the sequence of {14 component words, and if the parsing is successful, the entire compound word is converted into a noun or (8) Succeeding in analyzing the constituent elements of the compound word formed by special characters such as the hyphen, and converting the entire compound J6 into a noun, adjective, or When estimating a verb, prioritize these Shinagawa in the order of noun, adjective, and verb, and proceed with syntactic analysis (9)
In a natural language analysis system such as a machine translation system that has at least a system dictionary for translation, special characters such as hyphen r-J that combine simple J6 to form one compound word are added to the text to be analyzed by the syntactic analysis unit. When there is a compound word formed by characters, and the compound word does not exist in the dictionary of the system, special characters such as hyphens are replaced with spaces, and the compound word is made into a simple plural word with no special meaning. (10
) In a natural g-word analysis system such as machine translation that has at least a System 1 dictionary for translation, the morpheme generation unit
Compound words formed by special characters such as a hyphen "ichi" that combine words to form one compound word are looked up in a dictionary and parsed using the individual constituents of the compound word,
When translating the individual elements, between the individual constituent elements
The feature is that the above special characters are inserted into the . The following is a description based on embodiments of the present invention.

FIG. 1 is a configuration diagram for explaining an embodiment of a translation device equipped with a morphological analysis method, a syntactic analysis method, and a morphological generation method according to the present invention. In the figure, l is a CRT, 2 is a keyboard, and 3 is a OCR, 4 is input sentence a, 5 is spell check section, 6 is pre-editing section, 7 is translation main section, 8 is system dictionary,
9 is a grammar rule, 10 is a post-editing department, 11 is an output document, 12
is a printer.

Input sentences obtained by file input, keyboard input, or OCR input are sent to the spell check section 5 and the pre-edit section 6.
Perform pretreatment using The output sentence obtained by the translation main unit 7 can be edited by the post-editing unit 10 using translation information. The input and output sentences can be printed using printer 12.

The translation main unit (translation unit) 7 is roughly divided into four processes. FIG. 2 is a diagram showing the processing flow of the translation main body. The morphological analysis section performs dictionary lookup of the input text. The syntactic analysis section obtains information about individual words, performs parsing according to grammatical rules, and creates a tree structure from the analysis results. The conversion part transforms the tree structure of the input language into the tree structure of the output language. The generation unit translates the obtained tree structure node by node. The present invention pertains to a morphological analysis section, a syntactic analysis section, and a morphological generation section.

Here, the input text is English. A dictionary lookup is performed on the input text in the form S search analysis section = the result of the dictionary lookup is obtained and the process proceeds to the syntax analysis section.

Examples of special characters such as hyphens include hyphen 'slash'/'. Hereinafter, processing of unknown compound words will be explained using a specific example of a hyphenated word (a compound word connected by a hyphen).

First, the processing in the morphological analysis section will be explained.

(1) For example, let's take the word light-meal. Look up this from the system dictionary. Here, if there is no hyphen, that is, light-
You can also look it up in the dictionary as "meal" with just one finger.

As a result of the dictionary lookup, if there is any information, that information can be used to proceed with the subsequent translation process, but if not, it is an unknown word.
Some kind of processing must be done.

Azusa: A dictionary has at least the following information about a certain word. For a certain headword, it has at least 1) part of speech, 2) declension, 3) priority of part of speech, 4) translation, and 5) part of speech and conjugation of the translation.

These are usually described in the translation system dictionary a, but unknown words do not have any of these. Therefore, when translating, a temporary dictionary is created and information about the unknown word is written into it. Most of the unknown words in hyphenated languages are nouns, but they are lowered in priority to adjectives,
Information such as verbs may also be given. The translated words should remain alphabetical ('light-n+eal').
Nouns do not require a special ending, but adjectives do not require a special ending.
Add ``to'' to the verb. This information may be temporarily stored in a dictionary, and this information may be looked up in the dictionary and stored in memory, or may be written out to a file. The next step is parsing, which uses this information.

When the entire light-meal is not in the dictionary,
When we look up 'light'meal' in a dictionary, we get the following result, for example.

light adjective bright light noun light light verb light meal meal noun The obtained dictionary information may be stored in memory or may be exported to a file. This information is then passed to the next processing step, parsing, where this information is used. Further, at this time, it is also possible that individual words do not exist in the dictionary.

In that case, unknown word processing similar to (1) above is performed on the word. For light-meal, the first translation will be given as "light meal," but the morpheme generator may translate the hyphen in the original sentence as is, resulting in "light meal." Also, this hyphen may be half-width or full-width.

(3) Have eight double J's.Have both of the above (1) and (2) as a result of ff harm.Do you have these dictionary information in memory?
Or you can write it to a file. This information is then passed to the next processing step, parsing, where this information is used.

The following will explain the processing in the syntax analysis section.

Note that the processing in the syntax analysis unit will be explained on the assumption that (3) in the above explanation, that is, both static letters ↑+lI information are combined.

(1) Add to the hyphen (I-think- I-coIn) The c
company markCts its service
es with aplucky, I-think
-I-can spirit. The translation result when hyphen words are analyzed and the result is left as is is as follows.

``The company sells its services to markets where I think it canned alcohol.'' When the hyphen word is used as an unknown word, the translation result is as follows.

“The company is a vibrant T-think-I-ca
n%'f God and ship the service to the market. (2) Add to the hyphen (r-think-I-can-do-it)
The company markets its
services with aplucky, I-
think-I-can-do-it spirit.
``The company will ship its services to market with a cheerful person who thinks I can do it.''Here's a quick explanation of how hyphen parsing is done in practice. Explain by showing the rules.

A. Explanation of part-of-speech classification code prn (nominative pronoun). pro (objective pronoun),
aux (auxiliary verb). itl (transitive verb, which takes a phrase as its object), 1t2 (transitive verb, which takes that clause as its object).

B. Grammar code explanation SN (nominative noun phrase). ON (object noun phrase), VI (infinitive collocation), VC (finite predicate), O C (that clause, that can be omitted), SG (sentence not containing sentence-final clause B)
, TL (title, or noun phrase).

C6 grammar rules 1. 3G->SN vC2. 0C
-> SG 3. VI -> itl ○N4. V
I -> it2 QC5. VC->
aux V 16. VC-)VI 7. S N -> prn 8. ON -> pro However, the number at the beginning of the line is the rule number.

First, analyze the hyphen li+'j in a below from the end of the word. For simplicity, each word is assumed to have only the most representative part-of-speech classification. I-think- I-ca
In its current form, n is an unknown word that cannot be registered in dictionaries.

(1; Margin) With the given rules and part-of-speech sequence, this hyphenated word cannot be combined into a single grammatical code, resulting in a failure in analysis.

If you use this result as is to analyze the entire syntax, you will get the following bad translation results for the example sentence below.

The company markets its
services with aplucky, I
-think-I-can spirit. ``The company sells its service to a market where I think it canned alcohol.'' In this case, there is no point in trying to analyze the hyphen, so if it fails, it will be treated as an unknown word. . That is, I -think- I-
Let can be an unknown word. In this sentence, it is a noun modifying component related to spirit. In this case, we get the following translation.

``The company is a vibrant I-think- T-can
Deliver the service to the market in spirit. ” Next, for comparison, the hyphenated word b below is analyzed starting from the end of the word. For simplicity, each word is assumed to have only the most representative part-of-speech classification.

I-think- I-can-do-it is an unknown word that is not registered in the dictionary in its current form.

(Left below) Z+UZ O U+UZO ○ >>C/) ○ 〉 〉
of CO v tz-C-2 ma
In this case, it is grouped into SG. After all I
-think- I-can-do-it is translated as "I think I can do it," and it uses spirit as a noun modifying component.

The coIIlpany markets
jl. s Scrv. ices with ap
lucky, I-think- I-can-
do-it spirit. The r company will market its services with Kishin, who is energetic and thinks I can do it. ”At this time I −tt+ink
- The I-can-do-it part is determined to be a noun, adjective, or verb by the syntactic analysis unit, and is analyzed in the same order assuming priority.

It also has the beginning and end information of hyphenated words passed from the syntax analysis, and can insert hyphens when translating Japanese.

FIG. 3 is a flowchart showing the processing in the morphological analysis unit of an unknown compound word containing special characters such as a hyphen.

First, look up a dictionary. Next, determine whether it is in the dictionary. If the word is not in the dictionary, the entire word is processed as an unknown word. Next, look up each word in a dictionary. If there are any individual unknown words, process the unknown words. The next step is to store the dictionary information in memory or in a file.

FIG. 4 is a flowchart showing unknown word processing. First, create a temporary dictionary. Next, set the translation information,
Next, part of speech information is set, and then priority information is set. Then, look up the dictionary from the temporary dictionary.

FIG. 5 is a flowchart showing the processing in the syntactic analysis unit of a compound word with special characters such as a hyphen, which is an unknown word.

First, set the analysis start point to the end of the sentence and perform syntax analysis.

If the parsed sentence is at the beginning of a sentence, it ends, but if it is not at the beginning of a sentence, it advances one word. Next, it is determined whether it is an unknown hyphenated word or not, and if it is a hyphenated word, the hyphenated word is analyzed. If the analysis result is a failure, the hyphenated word is treated as an unknown word.

Brief Summary As is clear from the above explanation, according to the present invention, unknown word processing (morphological analysis processing), syntactic analysis processing, and translation processing of compound words using special characters such as hyphens can be performed more smoothly. , it is possible to obtain more understandable results.

[Brief explanation of drawings]

FIG. 1 is a block diagram for explaining an embodiment of a translation device equipped with a morphological analysis method, a syntactic analysis method, and a morpheme generation method according to the present invention, and FIG. 2 shows the processing of the translation main body of the translation device. 3 is a flowchart showing the processing in the morphological analysis unit of a compound word with special characters such as a hyphen, which is an unknown Mli. FIG. 4 is a flowchart showing unknown word processing, and FIG. is a flowchart showing processing in the syntactic analysis/folding section of a compound word with special characters such as a hyphen, which is an unknown word. {...CRT, 2...Keyboard, 3...OCR
, 4... Input document, 5... Spell check section, 6...
・Pre-editing department, 7... Translation body section, 8... System dictionary, 9... Grammar rules, 10... Post-editing department, l1...
- Output document, ■2...Printer. Figure 3 Figure 4 Figure 5

Claims (1)

[Claims] 1. In a natural language analysis system such as machine translation that has at least a system dictionary for translation, words are mutually combined to form one compound word in a text to be analyzed by a morphological analysis unit. When there is a compound word formed by special characters such as a hyphen "-", the compound word is looked up in the dictionary provided by the system, and if it does not exist in the dictionary, the entire compound word is treated as an unknown word and translated by parsing etc. A morphological analysis method characterized by registering data in a temporary translation dictionary used for processing. 2. In a natural language analysis system such as machine translation that has at least a system dictionary for translation, the text to be analyzed by the morphological analysis unit contains hyphens such as "-" that combine words to form one compound word. When there is a compound word formed by special characters, the compound word is looked up in a dictionary provided by the system, and when it does not exist in the dictionary, the individual constituent elements that make up the compound word are looked up in the dictionary, and those words are looked up in the dictionary. A morphological analysis method characterized in that information can be used for translation processing such as syntactic analysis. 3. In a natural language analysis system such as a machine translation system that has at least a system dictionary for translation, the text to be analyzed by the syntactic analysis unit contains characters such as hyphens "-" that combine words to form one compound word. When there is a compound word formed by special characters and the compound word does not exist in the dictionary of the system, the compound word is parsed using the sequence of constituent words, and if the parsing fails, the entire compound word is parsed. A syntactic analysis method that is characterized by processing the words as unknown words and proceeding with the syntactic analysis. 4. In a natural language analysis system such as a machine translation system that has at least a system dictionary for translation, the text to be analyzed by the syntactic analysis unit contains characters such as hyphens "-" that combine words to form one compound word. When there is a compound word formed by special characters and the compound word does not exist in the dictionary of the system, the compound word is parsed using the sequence of constituent words, and if the parsing is successful, the compound word is A syntactic analysis method that is characterized by proceeding with syntactic analysis by inferring that the whole is a noun, adjective, or verb. 5. In a natural language analysis system such as a machine translation system that has at least a system dictionary for translation, the text to be analyzed by the syntactic analysis unit contains characters such as hyphens "-" that combine words to form one compound word. When there is a compound word formed by special characters and the compound word does not exist in the dictionary of the system, the special characters such as hyphens are replaced with spaces and the compound word is made into a simple multiple word with no special meaning. A syntactic analysis method characterized by proceeding with syntactic analysis as a sequence of words. 6. In a natural language analysis system such as machine translation that has at least a system dictionary for translation, in the morpheme generation unit,
A compound word formed by special characters such as a hyphen "-" that joins words together to form a compound word is lexicographically looked up and parsed using the individual constituents of the compound word,
A morpheme generation method characterized in that when translating the individual constituent elements, the special characters are inserted between the individual constituent elements.