JP4186774B2 - Information extraction apparatus, information extraction method, and program - Google Patents

Description

  The present invention relates to an information extraction apparatus, an information extraction method, and a program for extracting information required by a user from a document and associating information in the document with a template indicating an attribute to be extracted.

  Conventionally, there have been disclosed in the following papers 1 and 2 as information extraction apparatuses for extracting information required by a user from a document.

  Paper 1: “Automatic Digest Generation of Electronic News”, Information Processing Society of Japan, Journal Vol. 36, no. 10 (1995), conference information is extracted from a news group for conference articles. For each attribute to be extracted (hereinafter referred to as an extracted attribute), feature information (such as a headline label word and a language expression pattern) is prepared and used for information extraction. These extracted attributes are flat templates such as “Title”, “Date”, “Location”, “Paper Deadline”, and “Article Type”.

  Similarly, Paper 2: “Collecting, classifying, and integrating information from wide-area networks based on ontology”, Information Processing Society of Japan, Vol. 38, no. 3 (1997), the integrated information is a flat template.

"Automatic digest generation of electronic news", Information Processing Society of Japan, Journal Vol. 36, no. 10 (1995) “Collecting, classifying, and integrating information from wide-area networks based on ontology” Transactions of Information Processing Society of Japan, Vol. 38, no. 3 (1997)

  However, the following problems arise when the above flat template is used.

(Problem 1)
For example, if it is desired to extract “the address of the venue” in addition to “the venue” as an item in Table 1 of the paper 1, the paper 1 adds “the address of the venue” to the item of Table 1 in the paper 1. It will be. However, the relationship between “the venue” and “the address of the venue” cannot be expressed. “Venue” indicates information about the place, so information such as “address” and “zip code” is also written nearby, but it cannot be expressed.

(Problem 2)
For another purpose, if you want to extract the address of a company or the mailing address of a contact, the location information such as the location name, zip code, and address becomes a similar extraction rule. Therefore, we want to reuse location information not only for the location of meetings, but also for the purpose of extracting company information and contact information. However, in the case of the structure as in paper 1, separate templates must be prepared, and the templates cannot be reused.

  The present invention has been made in view of the above-described problems of conventional information extraction apparatuses, and a main object of the present invention is a new and improved attribute extraction capable of reusing an existing template. An information extraction device, an information extraction method, and a program are provided.

In order to solve the above problems, according to a first aspect of the present invention, there is provided an information extraction apparatus for associating information appearing in an input document with attributes . The information extraction device (1) of the present invention stores an extracted knowledge storage unit in which a template composed of a set of one or more attributes to be extracted is stored, and the attributes and extracted knowledge of the attributes are stored in association with each other. (11), an extracted attribute determining unit (12) that refers to the extracted knowledge storage unit and associates the information extracted from the document with the attribute, and an output unit (13) that outputs the attribute and the value of the attribute. The attribute stored in the extracted knowledge storage unit (11) is characterized by referring to another related template .

  According to such a configuration, it is possible to perform attribute extraction in which an existing template can be reused by performing an operation for preferentially associating information appearing in the document with the extracted attribute in the same template. it can.

  In the information extraction apparatus of the present invention, the following applications are possible.

Extracting attribute determining section (12) extracts the named entities from the document, named entity extraction unit that adds the kind of unique representation extracted named entities and (121), referring to the extracted knowledge storage unit, unique Candidate attribute determination unit (123) that determines an attribute that corresponds to the type of the unique expression added by the expression extraction unit and an attribute that refers to the template including the attribute corresponding to the type of the specific expression as a candidate attribute for the specific expression Attribute determination for determining the template to which the determined candidate attribute most belongs for the specific expression extracted from the predetermined document range, and determining the candidate attribute belonging to the template as the attribute of the specific expression extracted from the document range Part (124) .

The predetermined document range may be a range divided by blank lines in the document , or may be a range in which lines where specific expressions appear are continuous .

The extracted knowledge storage unit (11) indicates that, when a template including an attribute to be extracted is referred to by another attribute and the predetermined condition is satisfied, the other attribute is not added to the candidate attribute. Information may be stored . In general, by avoiding attributes that do not appear in the document as candidate attributes, the number of candidate attributes increases, so that the amount of calculation does not increase and the extraction accuracy does not decrease.

The extraction attribute determining unit (12) further includes a character string element dividing unit (122) for extracting a character string element in the document as a character string element by separating the character string in the document with a line feed, a space, and a character mark. In the case of the range to be delimited, the candidate attribute determination unit may not perform the process of determining the candidate attribute for the specific expression extracted from the character string element .

In order to solve the above problems, according to a second aspect of the present invention, there is provided an information extraction method for associating information appearing in an input document with attributes . The information extraction method according to the present invention stores a document input step (S100) executed by a document input unit that receives a document as input, and a template composed of a set of one or more attributes to be extracted. An extracted attribute determining step (S110 to S140) executed by an extracted attribute determining unit that associates information extracted from a document with an attribute with reference to an extracted knowledge storage unit that is stored in association with the extracted knowledge, and an attribute and its attribute And an output step (S150) executed by the output unit that outputs the value of the attribute, and the attribute stored in the extracted knowledge storage unit refers to another related template .

  According to such a configuration, it is possible to perform attribute extraction in which an existing template can be reused by performing an operation for preferentially associating information appearing in the document with the extracted attribute in the same template. it can.

  In the information extraction method of the present invention, the following applications are possible.

The extraction attribute determination process refers to the specific expression extraction process executed by the specific expression extraction unit that extracts the specific expression from the document and adds the type of specific expression to the extracted specific expression, and the extracted knowledge storage unit . Executed by the candidate attribute determination unit that determines the attribute corresponding to the type of the specific expression added in the specific expression extraction step and the attribute that refers to the template including the attribute corresponding to the type of the specific expression as the candidate attribute for the specific expression a candidate attribute determining step (S130) of, for named entities extracted from the predetermined document range, determined candidate attribute determines the most belongs template, extracted candidate attributes belonging to the template from a document range specific And an attribute determination step (S140) executed by an attribute determination unit that determines the attribute of the expression .

The predetermined document range may be a range divided by blank lines in the document , or may be a range in which lines where specific expressions appear are continuous .

The extraction attribute determining step includes a character string element dividing step (S120) executed by a character string element dividing unit that extracts a character string element in a document as a character string element by dividing a character string in a document by a line feed, a space, and a punctuation mark. In the dividing step, when the character string element is delimited by a punctuation mark, the candidate attribute determination step may not be performed on the specific expression extracted from the character string element .

According to another aspect of the present invention, there are provided a program for causing a computer to function as the information extraction apparatus of the present invention, and a computer-readable recording medium on which the program is recorded . Here, the program may be described in any programming language. In addition, as a recording medium, for example, a CD-ROM, a DVD-ROM, a floppy disk (FD), a recording medium that is currently used as a recording medium capable of recording a program, or any recording that will be used in the future Media can also be employed.

  In the above description, the reference numerals and steps in parentheses attached to the constituent elements are merely shown as examples of corresponding constituent elements in the following embodiments and drawings for easy understanding. The invention is not limited to this.

  As described above, according to the present invention, an existing template can be reused by performing an operation for preferentially associating information appearing in the document with the extracted attribute in the same template. Attribute extraction can be performed.

  Exemplary embodiments of an information extraction apparatus, an information extraction method, and a program according to the present invention will be described below in detail with reference to the accompanying drawings. In the present specification and drawings, components having substantially the same functional configuration are denoted by the same reference numerals, and redundant description is omitted.

(First embodiment)
In the first embodiment, in order to solve the problems 1 and 2, it is expressed by a network structure in which feature information of a certain extraction attribute is in another template. Then, an operation for preferentially associating a plurality of pieces of information appearing close to each other in a document with extracted attributes in the same template is performed.

FIG. 1 is an explanatory diagram showing an information extraction apparatus according to the present embodiment.
As shown in FIG. 1, the information extraction apparatus 1 includes a document input unit 10, an extracted knowledge storage unit 11, an extraction attribute determination unit 12, and an output unit 13. Further, the extraction attribute determination unit 12 includes a specific expression extraction unit 121 , a character string element division unit 122, a candidate attribute determination unit 123, and an attribute determination unit 124. Further, the attribute determining unit 124 includes a block dividing unit 125, a template ranking unit 126, and an attribute calculating unit 127. Hereinafter, these components will be described in detail.

(Document input unit 10)
The document input unit 10 receives a text document designated by the user as input.

(Extracted knowledge storage unit 11)
The extracted knowledge storage unit 11 stores information about what the extraction attribute determination unit 12 extracts (extraction attributes) and how to extract (feature information for each extraction attribute). The feature information of the extracted attribute is expressed by a network structure indicating that it is in another template.

  FIG. 2 shows the structure of the extracted knowledge storage unit 11. Each template has “extracted attribute name (attribute name)”, “attribute value type”, “reference to another template”, “word that becomes the headword of the attribute (headword)”, “the attribute is Even if there is no entry word, it has an item “whether it is associated with an element in the document”. Each line in FIG. 2 represents feature information of the extracted attribute. The “attribute value type” item indicates whether the value to be included in the extracted attribute is a unique representation of a person name, an organization name, or an event name. The item “word to be a headword of the attribute (headword)” indicates a word described in the document as a headline of the attribute. The item “whether the attribute is associated with an element in the document even if there is no entry word” indicates whether an entry word is required when the element in the document is associated with the attribute. In the table, “Yes” is used for association, and “No” is used for no association.

  For example, in the first line of the event information template, the attribute value of the extracted attribute name “event name” is a pattern representing “event name”, and “name” is considered as the headword. If a character string that matches the pattern representing "event name" appears in the document, associate that character string with the extracted attribute name "event name" even if there is no corresponding headword before it. Indicates.

  In addition, in the fifth line of the event information template, the attribute value of the extracted attribute name “Venue” is shown in the place information template. “Venue” and “place” can be considered as headwords. Even if a word that matches the headword does not appear in the document, it is associated with the event information venue.

  Also, for example, the address of the event venue is represented by the “address” of the location information template by following the “reference to other template” item from the “venue” of the event information template. This is a dot like [template 1], [attribute name of template 1], [attribute name of template 2] ... [attribute name of template n], such as "event information. Venue. Address" Expressed in notation (n is called the depth of the template portion).

  In this, the part which points to a template is called a template part. For example, in “event information. Venue. Address”, “event information. Venue” ([template 1], [attribute name of template 1]) points to the location information template (template 2). It is called a template part. In this example, the depth of the template portion is 2.

  Also, if [attribute name of template n] is a reference to another template, [template 1]. [Attribute name of template 1]. [Attribute name of template 2]... Up to [attribute name of template n] are template parts. For example, the template part of “event information. Lecture” is “event information. Lecture”, and the depth of the template part is 2.

(Extraction attribute determination unit 12)
The extracted attribute determination unit 12 refers to the extracted knowledge storage unit 11 and associates the extracted information with the attribute. The extraction attribute determination unit 12 includes a specific expression extraction unit 121 , a character string element division unit 122, a candidate attribute determination unit 123, and an attribute determination unit 124. Further, the attribute determining unit 124 includes a block dividing unit 125, a template ranking unit 126, and an attribute calculating unit 127. Hereinafter, these components will be described in detail.

( Specific Expression Extraction Unit 121 )
The specific expression extraction unit 121 adds a tag to the specific expression in the input document. There are various types of tags such as a person name, an organization name, a job title, a place name, and an event name. As such a specific expression extraction unit 121 , for example, an apparatus described in “Comparison of Japanese and English in specific expression extraction” by Fukumoto et al. (Science Technical Report NLC 98-21 (1998)) can be used. The apparatus shown in the same document is merely an example for realizing the function of the named entity extraction unit 121 according to the present embodiment, and a detailed description thereof is omitted. For example, an organization name is extracted. To do this, a regular expression is used to extract parts such as “XX Co., Ltd.” and “Co., Ltd. XX” including “Co. For the event name, extract the part including “Symposium” or “Seminar” at the end. In the case of a person name or job title, a possible person name may be held in advance by a dictionary used in morphological analysis, and extracted by character string matching. It goes without saying that a device other than the device shown in the same document may be adopted in order to realize the function of the specific expression extraction unit 121 .

(Character string element division unit 122)
The character string element dividing unit 122 extracts a character string in the document as a range delimited by a line feed, a space, and a punctuation mark. If the character string element is delimited by a punctuation mark, it is regarded as a sentence, and the character string in that range is not processed by the candidate attribute determination unit 123 thereafter. This is because when a character string element is a sentence, it is easy to cause an association with an incorrect attribute unless processing such as parsing is performed.

(Candidate attribute determination unit 123)
The candidate attribute determination unit 123 refers to the extracted knowledge storage unit 11 to determine which attribute the specific expression extracted by the specific expression extraction unit 121 is considered to be.

(Attribute determination unit 124)
The attribute determination unit 124 determines the attribute of each unique expression in consideration of candidate attributes before and after each specific expression.

(Block division unit 125)
The block dividing unit 125 finds a range where the specific expression appears continuously in each line, and divides the range as a block.

(Template ranking unit 126)
The template ranking unit 126 calculates which template has the attribute corresponding to the unique expression of each block.

(Attribute calculation unit 127)
The attribute calculation unit 127 determines the attribute of each unique expression for each block.

(Output unit 130)
The output unit 130 outputs the attribute and the value of the attribute.

The information extraction apparatus 1 according to the present embodiment is configured as described above.
Next, the operation of the information extraction device 1 will be described.
FIG. 3 is a flowchart showing the operation of the first embodiment. Hereinafter, the operation will be described with reference to this flowchart. Here, as an example, a case where the document shown in FIG. 4 is received as an input will be described.

(Step S100)
The document input unit 10 receives a document as input. Here, as an example, a case where the document shown in FIG. 4 is received as an input will be described.

(Step S110)
The extraction attribute determination unit 12 activates the specific expression extraction unit 121 . The entire document is input to the specific expression extraction unit 121. Add a tag to a specific expression in a document. The result of tagging the document of FIG. 4 is shown in FIG. The line position item indicates the line number from the beginning, and the start position item indicates the number of characters from the beginning of each line. A headword is extracted as a character string that starts with a character string of two or more characters and includes:,], etc. for each line. This process can be performed by pattern matching. For example, the fourth line in FIG. The method for extracting headwords may include a dictionary related to headwords in addition to pattern matching, and is not limited to this method.

(Step S120)
The character string element dividing unit 122 receives a document as an input and delimits each character string element by a line feed symbol, a space symbol, and a punctuation mark. The result of dividing the document of FIG. 4 into character string elements is shown in FIG. As shown in FIG. 6, each character string element has items of a line position, a start position from the beginning of the character string element, a character string element, and a delimiter. For example, since the “first seminar” at the line position = 1 ends with a line feed, the delimiter is “line feed”. On the other hand, for “July 23” at the line position = 2, since the next character is a space symbol, the delimiter is “blank”.

(Step S130)
The candidate attribute determination unit 123 determines an attribute (candidate attribute) that is a candidate for association for each unique expression extracted in step S110. For each unique expression, the one that matches the “attribute value type” item in the extracted knowledge storage unit 11 is a candidate attribute.

FIG. 7 is a flowchart showing processing of the candidate attribute determination unit 123. The specific expressions obtained in step S110 are N ₁ , N ₂ ,..., N _m from the top. For example, in FIG. 5, since there are 17 unique expressions, m is 17.

(Step S1000)
A counter i indicating a specific expression to be processed is initialized to 1.

(Step S1010)
If i is less than or equal to m, go to step S1020. If not, the process ends (the process ends for all proper expressions).

(Step S1020)
The specific expression N _i is set as a processing target. For N _i, it is, it is determined whether or not has become part of the sentence. The string element corresponding to the position information of the N _i (row position items and start position items), found from the processing result of step S120. That is, row position items N _i is matched to the row position Item string element, and the start position items N _i - at the start entry of the string elements are 0 or more, finding the minimum one. For example, the character string element corresponding to N ₅ “XX city XX town 1-1” is “XX city XX town 1−1” on the fifth line in FIG. 1 ”.

(Step S1030)
If the delimiter item of the corresponding character string element is “punctuation”, it is regarded as a sentence and the process goes to step S1120 without being subjected to subsequent processing. Otherwise, go to step S1040. For example, in FIG. 5, the character string element corresponding to N ₈ “XX seminar” is “in this XX seminar, the purpose is to improve the ability that will be required in the future” in FIG. Is not subject to processing because the delimiter is "punctuation". On the other hand, all other unique expressions are processed because the corresponding character string elements are other than “punctuation”.

(Step S1040)
Comparing the N _i, the type entry or lemma item attribute values of the extracted knowledge memory unit 11. Type item of specific representation of N _i is if "entry word", go to step S1050. On the other hand, type item of specific representation of N _i is if other than the "entry word", go to step S1060.

(Step S1050)
It is checked whether or not there is a Ni _i specific expression item that matches the entry word item in the extracted knowledge storage unit 11. The matched attribute is set as a candidate attribute, and the process goes to step S1070. For example, in FIG. 5, “host” of N ₆ matches the entry item “event information. Host” in FIG. "Program" of N ₉ is to find words to match the "Event Information. Lecture" in FIG. 2.

(Step S1060)
The "type of named entity" item of N _i, the "type of the attribute value" item of the extracted knowledge memory unit 11, checks whether there is a match. The following conditions 1 and 2 are checked for those that match.

[Condition 1] If the value of the item “whether to associate the attribute even if there is no entry word” is “Yes”, the attribute is added to the candidate attribute. If not, check condition 2.
"Even if there is no entry word, whether or associate the attribute" [Condition 2] of the attribute with the value of the item is "NO", and the "type of unique expressions" item in the same row of the N _i is "entry word Is added and the candidate attribute of the specific expression matches the attribute. If not, don't add.
Go to step S1070.

FIG. 8 shows the candidate attributes of FIG. For example, since the “type of unique expression” item of N ₁ “1st XX seminar” is an event name, it matches “event information. Event name” in FIG. On the other hand, since the type of the specific expression of “July 23” of N ₂ is the date and time, it matches the “event information.date” and “lecture information.lecture time” in FIG. Add matching attributes as candidate attributes.

In addition, since the item “specific expression type” of N ₇ “Δ University” is an organization name, it matches “event information. Sponsor” and “organization information. Name” in FIG. Since the value of the item “whether the attribute is associated even if there is no headword” of “event information. Sponsor” is “No”, the unique expression on the same line is checked. Since the item of “type of proper expression” of N ₆ is “headword” and the candidate attribute is “event information. Sponsor”, “event information. Sponsor” is added as the candidate attribute of N ₇ .

(Step S1070)
If the attribute information does not exist, go to step S1120 to process the next specific expression. Otherwise, go to step S1080.

(Step S1080)
The candidate attributes determined in step S1050 or step S1060 are P _{i, 1} , P _{i, 2} ,..., P _{i, n} . For example, in FIG. 8, P _1,1 is “event information.event name”, P _2,1 is “event information, date”, and P _2,2 is “lecture information, lecture time”. A counter j indicating a candidate attribute to be processed is initialized to 1.

(Step S1090)
If j is n or less, go to Step S1100. If not, the process goes to step S1120 (the process ends for all candidate attributes).

(Step S1100)
Candidate attributes P _{i, j} are to be processed. The reference source of the template to which P _{i, j} belongs is found using the “reference to another template” item. Whether the attribute of the reference source is added to the candidate attribute is determined under the following conditions.
[Condition 1] If the value of the item “whether the attribute is associated even if there is no entry word” of the reference source attribute is “Yes”, the attribute is added to the candidate attribute. If not, check condition 2.
"Even if there is no entry word, whether or not associate the attribute" [Condition 2] of the reference source of the attribute with the value of the item is "NO", and the "type-specific expression" in the same line of N _i item " If there is a specific expression that is a “keyword” and the candidate attribute of the specific expression matches the attribute of the reference source, it is added. If not, don't add.

For example, the candidate attribute of "Taro Yamada" of N ₁₂ is a "person information. Name". There are “lecture information. Lecturer” and “organization information. President name” as attributes referring to the personal name information template. The item “whether the attribute is associated even if there is no headword” of “Lecture information. Speaker” is “Yes”, so this attribute is added to the candidate attribute. On the other hand, the item “whether or not to associate the attribute even if there is no headword” of “organization information. President name” is “No”, and the same line (line position = 9) of N ₁₂ has “unique” Since there is no unique expression whose “expression type” item is “entry word”, “organization information. President name” is not a candidate attribute.

Also, the added candidate attribute is recursively checked to see if it is referenced from another template, and step S1100 is executed. For example, in P _1,1 and P _2,1 , since the event information template is not referenced from other templates, there is no candidate attribute to be added. On the other hand, in P _{2 and 2} , the lecture information template is referenced from “event information. Lecture”. Therefore, “event information. Lecture. Lecture time” is added to the candidate attribute. For the added “event information. Lecture. Lecture time”, the event information template is not referenced from other templates, so there are no more candidate attributes to add. FIG. 8 shows candidate attributes added in step S1100.

(Step S1110)
Add 1 to j for calculation with the next candidate attribute. The process returns to step S1090.

(Step S1120)
Add 1 to i to calculate with the following specific expression. The process returns to step S1010.

The processing of the candidate attribute determination unit 123 has been described above.
Step S140 and subsequent steps in FIG. 3 will be described again.

(Step S140)
The attribute determining unit 124 determines an attribute corresponding to each unique expression from the candidate attributes determined in step S130. When determining an attribute, a block is defined for each area where the specific expression appears continuously in each line, and which template is associated with each block is determined. For example, in FIG. 4, since the continuation of the unique expression is interrupted at the 6th and 10th lines, the 1st to 5th lines, the 7th to 9th lines, and the 11th to 12th lines become individual blocks. In the first to fifth lines, the unique expressions N ₁ to N _{8 in} FIG. 8 belong to one block.

In order to determine which template attribute the specific expression of this range is associated with, one point is assigned each time the template portion is listed as a candidate attribute. The attributes in the high score template are preferentially associated with the specific expressions. For example, in the blocks from N ₁ to N ₈ , one point is assigned to event information at N ₁ . In N _2, 1 points to the event information, event information. One point is assigned to the lecture and one point is assigned to the lecture information. Similarly, in _{N 3,} 1 points to the location information, event information. One point at the venue, personal information. One point in the residence, lecture information. Speaker. One point in the place of residence, event information. Lecture. Speaker. One point in the place of residence, organization information. Assign a point to the location.

Finally, the sum of the blocks from N ₁ to N _8, the event information is 4 points, the event information. 3 venues, event information. Lecture. Speaker. 3 points of residence, 3 points of location information, personal information. Three points of residence, lecture information. Speaker. Three places of residence, organization information. 3 locations, event information. One lecture, event information. Lecture information. Speaker. ID. One affiliation organization, one lecture information, one organization information, identification information. One point of belonging organization, personal information. ID. One point of belonging organization, lecture information. Speaker. ID. Your organization becomes 1 point.

Next, among the candidate attributes N ₁ to N ₈ , the corresponding attribute is determined. Since the event information template has the highest score, the candidate attribute including the event information template is preferentially assigned to the attribute. Alternatively, it is also possible when determining the attributes corresponding to N _i, priority attributes of a template containing the attributes corresponding to the N _i-1. as a result,
N ₁ : Event information. Event name N ₂ : Event information. Date N ₃ : Event information. venue. Building name N ₄ : Event information. venue. Postal code N ₅ : Event information. venue. Address N ₆ : Event information. Organizer N ₇ : Event information. Organizer N ₈ : No candidate.

In order to perform such processing, FIG. 9 is a flowchart showing the processing of the block dividing unit 125 of the attribute determining unit 124. The specific expressions obtained in step S110 are denoted as N ₁ , N ₂ , and N _m from the top. It is assumed that a variable indicating the specific expression at the head of the block is start_block and a counter indicating the specific expression to be processed is i.

(Step S1200)
Initialize i to 1 and start_block to 0.

(Step S1210)
If i is less than or equal to m, go to step S1220. If not, the process goes to step S1250 (block calculation processing ends for all unique expressions).

(Step S1220)
The specific expression _Ni is used as a processing target. If start_block is 0, start_block is set to i.

(Step S1230)
If i is 1 or more and N _i row position−N _i−1 is 2 or more, it is regarded as the end of the block, and the process goes to step S1260. Otherwise, go to step S1240.

(Step S1240)
Add 1 to i to calculate with the following specific expression. The process returns to step S1210.

(Step S1250)
The template weight determination process in the block shown in step S1300 and subsequent steps is performed, and the process ends.

(Step S1260)
The template weight determination process in the block shown in step S1300 and subsequent steps is performed.

(Step S1270)
The start_block is initialized to 0, and the process returns to step S1240.

  FIG. 10 is a flowchart showing the operation of the template ranking unit 126 for determining the template weight in the block. It is assumed that a variable representing the last specific expression of the block is end_block and a counter indicating the specific expression to be processed is i2.

(Step S1300)
Set end_block to i. Set i2 to start_block.

(Step S1310)
If i2 is less than or equal to end_block, go to step S1320. Otherwise, go to step S1370.

(Step S1320)
Let N _i2 candidate attributes be P _i2,1 , P _i2,2 ,..., P _{i2, n} . A counter j2 indicating a candidate attribute to be processed is initialized to 1. The block information temporary storage unit is initialized. The block information temporary storage unit is composed of a template part item of candidate attributes of the block and its score.

(Step S1330)
If j2 is less than or equal to n, go to step S1340. Otherwise, go to step S1360 (processing is completed for all candidate attributes).

(Step S1340)
Candidate attributes P _{i2, j2} are to be processed. If the template portion of P _{i2, j2} exists in the template information temporary storage unit, 1 is added to the number of appearances of the corresponding template portion. The number of points to be added is not limited to 1 and may be 1 / n. In this example, it will be described as 1. If it does not exist, the template part is added to the template part item, and the number of appearances is set to 1 (or 1 / n). Go to step S1350. In this example, it will be described as 1.

(Step S1350)
Add 1 to j2 for calculation with the next candidate attribute. It returns to step S1330.

(Step S1360)
Add 1 to i2 to calculate with the following specific expression. The process returns to step S1310.

(Step S1370)
The attribute determination process in the block shown in step S1400 and subsequent steps is performed, and the process returns to the attribute determination process in step S1250 or step S1260.

  FIG. 11 is a flowchart showing the operation of attribute determination processing in a block. The counter indicating the specific expression to be processed is i3, and the counter indicating the candidate attribute to be processed is j3.

(Step S1400)
i3 is set to start_block.

(Step S1410)
If i3 is less than or equal to end_block, go to step S1420. Otherwise, the process ends and returns to step S1370.

(Step S1420)
A counter j3 indicating a candidate attribute to be checked is initialized to 1.

(Step S1430)
If j3 is less than or equal to n, go to step S1440. If not, since the processing is completed for all candidate attributes, the process goes to step S1510.

(Step S1440)
Let T ₁ , T ₂ ,..., T _t be template portions of the block information temporary storage unit. A counter k indicating the template portion to be processed is initialized to 1.

(Step S1450)
If k is equal to or less than t, the process proceeds to step S1460. Otherwise, go to step S1490 (processing is completed for all template portions).

(Step S1460)
It is checked whether T _k matches the template portion of P _{i3, j3} . If they match, go to step S1470. Otherwise, go to step S1480 to calculate the next template part.

(Step S1470)
The score of P _{i3, j3} is calculated by the following formula (w ₀ is a weight).
[ _Points of P _{i3, j3} ] = [ _Points of T _k ] + [Points calculated in step S1490] × w ₀
Go to step S1490 to calculate [the number of points calculated in step S1490].

(Step S1480)
Add 1 to k for calculation in the next template part. It returns to step S1450.

(Step S1490)
A candidate attribute having the same template part as the attribute associated with the specific expression N _{i3-1 immediately} before N _i3 is preferentially _treated . _If the template portion of P _{i3, j3} is completely equal to the template portion of the attribute corresponding to N _i3-1 , [the score calculated in step S1490] is set to 1. If the template part matches forward, the depth p of the matched part and the depth r of the template part of the attribute corresponding to the template part depth q and N _i3-1 of P _{i3, j3} are (0. 8) ^{Set to max (q, r) -p} . For example, P _{3 and 2} are “event information. Venue, building name”, P _{3 and 6} are “event information. Lecture, lecturer, residence, building name”, and the attribute corresponding to N ₂ is “event information”. Then, the number of points added to P _3,2 is (0.8) ^2-1 . This is because the matching part of the attribute template parts corresponding to P ₃ and ₂ and N ₂ is “event information”, and the depth of the template part of depth 1 and P ₃ and 2 is 2. On the other hand, the number of points added to P ₃ and ₆ is (0.8) ^4-1 . This is because the matching part of the template parts of the attributes corresponding to P ₃ , ₆ and N ₂ is “event information”, and the depth of the template part of 1, P ₃ , ₆ is 4. Go to step S1500. However, if i3-1 is 0, [the number of points calculated in step S1490] is set to 0.

(Step S1500)
In order to calculate with the next candidate attribute, 1 is added to j3. It returns to step S1430.

(Step S1510)
Of the points of P _i3,1 , P _i3,2 ,..., P _{i3, n} , the largest one is determined as the attribute of N _i3 . Go to step S1520.

(Step S1520)
Add 1 to i3 to calculate with the following specific expression. The process returns to step S1410. As mentioned above, in the block from N ₁ to N ₈ , if w ₀ = 1,
N ₁ : Event information. Event name (4 points)
N ₂ : Event information. Date (4 points + 1.0 x 1 point)
N ₃ : Event information. venue. Building name (3 points + 0.8 x 1 point)
N ₄ : Event information. venue. Zip code (3 points + 0.8 x 1 point)
N ₅ : Event information. venue. Address (3 points + 0.8 x 1 point)
N ₆ : Event information. Organizer (4 points + 0.8 x 1 point)
N ₇ : Event information. Organizer (4 points + 1.0 x 1 point)
N ₈ : No candidate

Further, in the block from _{N 9} to _{N 13,} the event information. 3 lectures, 1 event information, 2 lecture information, 1 personal information, lecture information. One point of lecturer, event information. Lecture. 1 speaker, 1 organization information, identity information. One point of belonging organization, personal information. ID. One point of belonging organization, lecture information. Speaker. ID. One point of organization, event information. Lecture information. Speaker. ID. Your organization becomes 1 point. And upper event information. Candidate attributes including lectures are mapped as follows.
N ₉ : Event information. Lecture (3 points)
N ₁₀ : Event information. Lecture. Lecture time (3 points + 1.0 x 1 point)
N ₁₁ : Event information. Lecture. Title (3 points + 1.0 x 1 point)
N ₁₂ : Event information. Lecture. Speaker. Name (1 point + 0.8 x 1 point)
N ₁₃ : Event information. Lecture. Speaker. ID. Affiliated organization. Name (1 point + 0.64 x 1 point)
The blocks from N ₁₄ to N ₁₇ are almost the same.

(Effects of the first embodiment)
As described above, according to the present embodiment, knowledge for associating a specific expression with an extracted attribute is expressed by a network structure in which feature information of a certain extracted attribute exists in another template. Then, by performing an operation for preferentially associating information that appears in the document in the vicinity with the extracted attribute in the same template, it is possible to perform attribute extraction that can reuse the existing template.

(Second Embodiment)
In the first embodiment, when referring to another template, all attributes in the reference template are candidate attributes. For example, when an address appears in a document, the address of the speaker at the event (event. Lecture. Speaker. Address) was also a candidate attribute. However, in general, since the address of the speaker is not described in the document indicating the lecture of the event, making such attributes as candidate attributes increases the number of candidate attributes, which increases the amount of calculation and increases the extraction accuracy. Will be reduced. Therefore, in the second embodiment, candidate attributes can be limited when they are referenced from a certain template.

  The configuration of the information extraction apparatus according to the second embodiment is substantially the same as that of the first embodiment. However, an item for not adding the attribute to the candidate attribute is added to the extracted knowledge storage unit 11 according to the template of the reference source. FIG. 12 shows an example of the extracted knowledge storage unit 11. The rightmost item in the extracted knowledge storage unit 11 is an item for not adding an attribute to a candidate attribute when a template is referenced. The value of this item is the attribute name of another template. For example, in “person information. Residence” in FIG. 12, “lecture information. Lecturer” is set in this item. This is because “person information. Place of residence” is an attribute referenced from other templates (for example, lecture information. Lecturer. Place of residence, organization information. President. Place of residence, event. Lecture. Lecturer. Residence). Talk) information. Indicates that attributes that have a speaker as a part of the template part (for example, lecture information, lecturer, residence, event. Lecture, lecturer, residence) are not set as candidate attributes.

  The operation of the information extracting apparatus according to this embodiment is different from that of the first embodiment in that the candidate attribute determination unit 123 is the same as that of the first embodiment.

  The following condition 0 is added to step S1100 of the first embodiment in FIG. Thus, step S2000 is performed instead of step S1100 in FIG.

(Step S2000)
Candidate attributes P _{i, j} are to be processed. Other templates referencing the template name of P _{i, j} are found using the “reference to other templates” item. Whether the attribute of the reference source is added to the candidate attribute is determined under the following conditions.
[Condition 0] If there is a value in the item “not to be a candidate attribute when referenced from another template” and the value matches the attribute of the reference source, it is not added to the candidate attribute. If not, check condition 1 and condition 2.
[Condition 1] If the value of the item “whether the attribute is associated even if there is no entry word” of the reference source attribute is “Yes”, the attribute is added to the candidate attribute. If not, check condition 2.
"Even if there is no entry word, whether or not associate the attribute" [Condition 2] of the reference source of the attribute with the value of the item is "NO", and the "type-specific expression" in the same line of N _i item " If there is a specific expression that is a “keyword” and the candidate attribute of the specific expression matches the attribute of the reference source, it is added. If not, don't add.

  Also, for the added candidate attribute, the “reference to another template” item is recursively referenced. However, when checking the condition 0, the “candidate attribute when it is referenced from another template” Take over the value of the item of “Nothing” and check it.

For example, the candidate attribute of N ₅ “XX city XX town 1-1” is “location information. Building name”. As attributes referring to the place information template, there are “person information. Place of residence”, “organization information. Location”, and “event information. Place”. Among these items, “Personal information. Resident” and “Organization information. Since “person information. Place of residence”, “organization information. Location”, and “event information. Venue” do not include “lecture information. Lecturer”, condition 0 is satisfied.

  However, considering the templates that refer to them, “Lecture Information. Lecture. Speaker. Residential” for “Personal Information. Residential” is not considered as a candidate attribute when referenced from other templates. Matches the “thing” item. Therefore, “lecture information. Lecture. Lecturer. Residence” is not set as a candidate attribute because condition 0 is not satisfied.

  Similarly, for “organization information. Address”, as a template referring to this, “lecture information. Lecture. Lecturer. Identity. Affiliation organization. Address” is not a candidate attribute because condition 0 is not satisfied. .

As a result, FIG. 13 shows an example of candidate attributes. Compared to FIG. 8, there are no candidate attributes starting with “Event.Lecture.Lecturer.Residence” and “Lecture information.Lecturer.Residence” with candidate attributes of N _{3 to} N ₅ .

(Effect of the second embodiment)
As described above, in this embodiment, in general, by not making an attribute that does not appear in a document a candidate attribute, the number of candidate attributes increases and the amount of calculation does not increase or the extraction accuracy does not decrease. You can

  The preferred embodiment of the information extraction apparatus according to the present invention has been described above with reference to the accompanying drawings, but the present invention is not limited to such an example. It will be obvious to those skilled in the art that various changes or modifications can be conceived within the scope of the technical idea described in the claims, and these are naturally within the technical scope of the present invention. It is understood that it belongs.

  For example, the following applications are possible.

(1) In the first embodiment, the attribute corresponding to each unique expression is narrowed down to one, but it may be better to list the second and third candidates. Therefore, a plurality of candidates can be presented to the user by specifying a threshold value for the number of candidate attributes to be output.

(2) In the first embodiment, a text document is taken as an example of input to the document input unit 10, but it can also be extended to an HTML document. For this purpose, in the character string element dividing process, in addition to the line position, when an HTML document is actually displayed on the browser, a process for extracting a range displayed in one line is performed. For example, even if the range surrounded by the tags <tr> and </ tr> representing each row of the table continues, it is displayed in one line unless there is a tag such as <br> or <p>. This can be analyzed by using the HTML tag analysis function of a conventional browser.

(3) In the above embodiment, the process up to associating attributes with each unique expression has been described. After this processing, for example, in the first embodiment, processing for detecting that N ₁₀ to N ₁₃ and N ₁₄ to N ₁₇ represent different lectures is performed. If there is an error in the process of associating the attribute with the proposed specific expression during the process, and the system determines that the correct result cannot be obtained in the subsequent processes, the process of associating the attribute with the specific expression again May be performed.

(4) When the character string element is a sentence, erroneous extraction is likely to occur unless processing such as syntax analysis is performed. Therefore, the part that is the sentence is not processed by the candidate attribute determination unit. However, it can be combined with a method of determining candidate attributes by performing syntax analysis or the like.

(5) In the above embodiment, the template is in a tabular form. However, even if it is described in a ternary format using an ontology description language such as RDFS (Resource Description Framework Schema), for example. Good.

(6) In step S1470, the addition is performed only for the template part that is completely matched. However, the addition may be performed even when a part of the template part is matched.

(7) In step S140, the block is divided for each range where the unique expressions are continuous, but the block may be divided for each blank line.

  The present invention is applicable to an information extraction apparatus, an information extraction method, and a program for extracting information required by a user from a document and associating information in the document with a template indicating an attribute to be extracted.

It is a block diagram which shows schematic structure of the information extraction apparatus concerning 1st Embodiment. 4 is an explanatory diagram illustrating an example of an extracted knowledge storage unit 11. FIG. It is a flowchart which shows operation | movement of the information extraction apparatus concerning 1st Embodiment. 4 is an explanatory diagram illustrating an example of an input document input to a document input unit 10. FIG. It is explanatory drawing which shows the example of an output of the specific expression extraction part 121. FIG. It is explanatory drawing which shows the example of an output of the character string element division part 122. FIG. It is a flowchart which shows operation | movement of the candidate attribute determination part 123. FIG. It is explanatory drawing which shows an example of a candidate attribute. 5 is a flowchart showing the operation of a block dividing unit 125. 5 is a flowchart showing the operation of a template ranking unit 126. It is a flowchart which shows operation | movement of the attribute calculation part 127. 4 is an explanatory diagram illustrating an example of an extracted knowledge storage unit 11. FIG. It is explanatory drawing which shows an example of a candidate attribute.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Information extraction apparatus 10 Document input part 11 Extracted knowledge memory | storage part 12 Extracted attribute memory | storage part 13 Output part 121 Specific expression extraction part 122 Character string element division part 123 Attribute information determination part 124 Attribute determination part 125 Block division part 126 Template ranking part 127 Attribute calculation part

Claims (10)

In an information extraction device that associates information appearing in an input document with attributes,
A document input unit that accepts a document as input;
An extracted knowledge storage unit that stores a template composed of a set of one or more attributes to be extracted, and stores the attribute and the extracted knowledge of the attribute in association with each other;
An extracted attribute determining unit that refers to the extracted knowledge storage unit and associates information extracted from the document with the attribute;
An output unit for outputting the attribute and the value of the attribute;
Including
The attribute stored in the extracted knowledge storage unit refers to another related template, and the extracted knowledge of the attribute stored in the extracted knowledge storage unit includes the type of attribute,
The extraction attribute determination unit
A named entity extraction unit for extracting a named entity from the document and adding the type of the named entity to the extracted named entity;
With reference to the extracted knowledge storage unit, the attribute associated with the attribute type that matches the type of the specific expression added by the specific expression extraction unit is determined as a candidate attribute for the specific expression , A candidate attribute determination unit that determines an attribute that refers to a template including the attribute as a candidate attribute for the specific expression;
Attribute determination for determining the template to which the determined candidate attribute belongs most for the specific expression extracted from a predetermined document range, and determining the candidate attribute belonging to the template as the attribute of the specific expression extracted from the document range Part,
An information extraction device characterized by including:   The information extracting apparatus according to claim 1, wherein the predetermined document range is a range divided by blank lines in the document.   2. The information extracting apparatus according to claim 1, wherein the predetermined document range is a range in which lines where the specific expression appears are continuous. The extracted knowledge storage unit stores, when a template including the attribute to be extracted is referred to by another attribute, a condition that is satisfied by one of the other attributes not added to the candidate attribute. The information extraction device according to claim 1, wherein the information extraction device is a feature. The extraction attribute determining unit includes a character string element dividing unit that extracts a character string element as a character string element by separating a character string in the document with a line feed, a space, or a punctuation mark.
When the character string element is in a range delimited by punctuation points, the candidate attribute determination unit is configured not to perform a process of determining a candidate attribute for the specific expression extracted from the character string element. The information extraction device according to claim 1. A template configured by a set of one or more attributes to be extracted, and an extracted knowledge storage unit that stores the attribute and the extracted knowledge of the attribute in association with each other; a document input unit; , An information extraction method comprising an extraction attribute determination unit and an output unit, and associating information appearing in an input document with an attribute,
The extracted attribute determining unit includes a specific expression extracting unit, a candidate attribute determining unit, and an attribute determining unit,
And Rubun document input step receiving as an input the document input unit is a document,
The extracted attribute determination unit refers to the extraction knowledge storage unit, and Extraction attribute determining step the information extracted from the document that associates the attribute,
The output unit and the attributes and output step you output the value of the attribute,
Including
The attribute stored in the extracted knowledge storage unit refers to another related template, and the extracted knowledge of the attribute stored in the extracted knowledge storage unit includes the type of attribute,
The extraction attribute determination step includes
Wherein the named entity extraction section extracts the specific expression from within the document, the extracted unique expressions above named entity types added unique expression extraction step,
The candidate attribute determination unit refers to the extracted knowledge storage unit, and determines the attribute associated with the attribute type that matches the type of the specific expression added in the specific expression extraction step with respect to the specific expression. and it determines as a candidate attribute, the attributes that reference template containing the attributes, and candidate attribute determination step that determine as a candidate attribute for the named entity,
For the specific expression extracted from the predetermined document range by the attribute determination unit, the template to which the determined candidate attribute belongs most is determined, and the candidate attribute belonging to the template is the attribute of the specific expression extracted from the document range. and attribute determination step that determine as,
An information extraction method characterized by including:   7. The information extraction method according to claim 6, wherein the predetermined document range is a range divided by blank lines in the document.   7. The information extraction method according to claim 6, wherein the predetermined document range is a range in which lines where the specific expression appears are continuous. The extracted attribute determining step includes a character sequence component division unit newline character strings in the document, blank, the extraction to Rubun string element dividing step ranges separated by punctuation mark as a string element,
The process according to any one of claims 6 to 8, wherein, when the character string element is delimited by a punctuation mark, the specific attribute extracted from the character string element is not processed by the candidate attribute determination step. Information extraction method described in 1.   A program for causing a computer to function as the information extraction device according to any one of claims 1 to 5.

Images