CN111831783B - Method for extracting chapter-level relation - Google Patents
Method for extracting chapter-level relation Download PDFInfo
- Publication number
- CN111831783B CN111831783B CN202010644404.XA CN202010644404A CN111831783B CN 111831783 B CN111831783 B CN 111831783B CN 202010644404 A CN202010644404 A CN 202010644404A CN 111831783 B CN111831783 B CN 111831783B
- Authority
- CN
- China
- Prior art keywords
- entity
- representation
- document
- sentence
- abstract semantic
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title abstract description 15
- 238000000605 extraction Methods 0.000 claims abstract description 31
- 230000002457 bidirectional effect Effects 0.000 claims abstract description 17
- 238000012545 processing Methods 0.000 claims abstract description 14
- 239000013598 vector Substances 0.000 claims description 74
- 230000003993 interaction Effects 0.000 claims description 68
- 239000011159 matrix material Substances 0.000 claims description 50
- 230000007246 mechanism Effects 0.000 claims description 21
- 238000011176 pooling Methods 0.000 claims description 7
- 239000000203 mixture Substances 0.000 claims description 6
- 238000006243 chemical reaction Methods 0.000 claims description 3
- 238000004364 calculation method Methods 0.000 abstract description 8
- 238000003058 natural language processing Methods 0.000 abstract description 2
- 230000006870 function Effects 0.000 description 11
- 238000013528 artificial neural network Methods 0.000 description 6
- 238000013135 deep learning Methods 0.000 description 4
- 230000000694 effects Effects 0.000 description 3
- 238000010801 machine learning Methods 0.000 description 3
- 238000013527 convolutional neural network Methods 0.000 description 2
- 230000014509 gene expression Effects 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- 230000004913 activation Effects 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000000306 recurrent effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012549 training Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/31—Indexing; Data structures therefor; Storage structures
- G06F16/313—Selection or weighting of terms for indexing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/35—Clustering; Classification
- G06F16/353—Clustering; Classification into predefined classes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/36—Creation of semantic tools, e.g. ontology or thesauri
- G06F16/367—Ontology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/30—Semantic analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/044—Recurrent networks, e.g. Hopfield networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Computational Linguistics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Databases & Information Systems (AREA)
- General Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Biomedical Technology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Molecular Biology (AREA)
- Evolutionary Computation (AREA)
- Biophysics (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Animal Behavior & Ethology (AREA)
- Machine Translation (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides a chapter-level relation extraction method, relates to the technical field of natural language processing, and mainly solves the technical problems of calculation resource consumption and logic reasoning between a target entity and a non-target entity aiming at chapter-level documents. The invention comprises the following steps: inputting a document to be processed, wherein the document is a chapter-level document; processing the document based on bidirectional attention constraint to obtain abstract semantic representations of entities and sentences, wherein the abstract semantic representations have global information and logical reasoning information; and judging the relation type of the target entity pair in the document based on the abstract semantic representation. The developer can efficiently and accurately extract the chapter-level relation by using the method disclosed by the invention, and simultaneously solves two main problems of chapter-level relation extraction, namely the problem of calculation cost caused by traversing all entities to generate alternative samples and the problem of logic reasoning between target entities and non-target entities.
Description
Technical Field
The invention relates to the technical field of natural language processing, in particular to a chapter-level relation extraction method in the technical field of bidirectional attention constraint and graph convolution neural network intersection. The invention obtains abstract representation of entity and document by constraint of bidirectional attention mechanism, and uses chapter-level relation extraction technique of graph convolution neural network for logical reasoning.
Background
The relation extraction is based on given context and two target entities, and judgesTwo target entitiesWhat the relationship between them is. Relationship extraction is one of the very important techniques for constructing large-scale knowledge-graphs, and can also be used to assist some downstream tasks, such as question-answering systems, etc. The research on relation extraction is mainly divided into two directions, one direction is based on a traditional machine learning thought for constructing a large number of artificial feature engineering, and the other direction is a deep learning method for building a neural network.
In traditional machine learning solutions, some researchers construct different features for an entire sentence and for a given target entity pair. The solution to traditional machine learning is very dependent on feature engineering and expertise, so that a model needs a large amount of manpower to construct features, meanwhile, the generalization capability of the model can greatly fluctuate along with the change of a data set, and in recent years, a large number of researchers use a deep learning method to solve the problem of relation extraction. In terms of deep learning, researchers have focused their research directions on the application of Convolutional Neural Networks (CNNs), recurrent Neural Networks (RNNs), and attention mechanisms, as well as on improvements to the above models, thereby also yielding a number of relevant solutions based on deep learning approaches.
The existing task definition and method for extracting the relation are all tasks based on sentence level, namely, aiming at a sentence as a context, aiming at two target entities appearing in the current sentence, the relation of the target entities is judged. However, in a practical application scenario, the text processed in most cases is not just as simple at the sentence level, relatively speaking, a text with one chapter is more common, a large number of entities appear in each chapter, and at the same time, each target entity appears multiple times in one document, so some classical methods in relation extraction at the sentence level are not available in this case, such as entity location vectors, PCNN, and so on. Meanwhile, the length of the text at the article level is often hundreds or even thousands, which is far greater than that of the text at the sentence level, the text at the overlength causes difficult extraction of the information of the whole document, the information is depended on for a long distance, and good representation and interaction of the entity are difficult, so that the effect of extracting the model by the chapter-level relation is greatly reduced. Meanwhile, because a large number of entities exist, each combination of the entities should become a potential target entity pair, and by traversing all the entity combinations in each article, a sentence and target entity sample is constructed, and a sentence-level relation extraction model is applied, so that the calculation data consumption is greatly increased. More importantly, due to the increase of the text length, a large number of entities exist in the input document, and the relationship between two target entities is obtained through not only direct interaction but also logic reasoning among non-target entities, and can be obtained through one or more times of reasoning among the entities, so that the accurate classification result cannot be obtained for the relationship needing reasoning.
Disclosure of Invention
One of the purposes of the invention is to provide a chapter-level relation extraction method, which solves the technical problems of calculation resource consumption and logic reasoning between target entities and non-target entities aiming at chapter-level problems in the prior art. Numerous advantageous effects can be achieved in the preferred embodiments of the present invention, as described in detail below.
In order to achieve the above purpose, the present invention provides the following technical solutions:
the invention relates to a chapter-level relation extraction method, which comprises the following steps:
inputting a document to be processed, wherein the document is a chapter-level document;
processing the document based on bidirectional attention constraint to obtain abstract semantic representations of entities and sentences, wherein the abstract semantic representations have global information and logical reasoning information;
and judging the relation type of the target entity pair in the document based on the abstract semantic representation.
Further, the processing the document based on the bidirectional attention constraint to obtain an abstract semantic representation of an entity and a sentence includes:
obtaining a representation of each entity and a representation of each sentence in the document;
acquiring a global entity interaction matrix based on the representation of each entity;
calculating the global entity interaction matrix and the representation of each sentence based on a first attention mechanism to obtain the representation of each sentence based on entity attention;
calculating the representation of each sentence based on the entity attention based on the graph convolution network to obtain the representation of each sentence with logic reasoning information;
based on a second attention mechanism, calculating the global entity interaction matrix and the representation of each sentence with logical reasoning information to obtain a new global entity interaction matrix with global information and logical reasoning information, and taking the new global entity interaction matrix as the abstract semantic representation of the entity and the sentence.
Further, the obtaining a representation of each entity and a representation of each sentence in the document includes:
acquiring an abstract semantic representation of the document;
a representation of each entity and a representation of each sentence in the document is obtained based on the abstract semantic representation of the document.
Further, the obtaining a representation of each sentence in the document based on the abstract semantic representation of the document includes:
and extracting the composition of each sentence from the abstract semantic representation of the document according to the starting position and the ending position of each sentence, carrying out maximum pooling processing on the composition of each sentence, and taking a sentence vector obtained after the maximum pooling processing as the representation of each sentence.
Further, the obtaining a representation of each entity in the document based on the abstract semantic representation of the document includes:
and determining words forming the entities according to the positions of each entity, calculating an average word vector of the words, taking the average word vector as an entity vector, and taking the average vector of the entity vectors of the same entity as the representation of each entity.
Further, the obtaining the abstract semantic representation of the document includes:
performing word vector conversion on the document to obtain a word vector matrix of the document;
and carrying out bidirectional LSTM operation on the word vector matrix to obtain the abstract semantic representation of the document.
Further, the obtaining a global entity interaction matrix based on the representation of each entity includes:
information interaction is carried out on every two entities, and two entity vectors are converted into one entity interaction vector;
and forming a global entity interaction matrix by all entity interaction vectors.
Further, the matrix weights adopted by the first attention mechanism and the second attention mechanism are the same.
Further, the determining the relationship type of the target entity pair in the document based on the abstract semantic representation includes:
performing column-wise weighting operation on the new global entity interaction matrix to obtain a global entity interaction vector;
and inputting the global entity interaction vector into a preset classification function for distinguishing the relationship types of the entity pairs to obtain a classification result, and taking the classification result as the relationship type of the target entity pair in the document.
Further, the classification function employs a softmax function.
The chapter-level relation extraction method provided by the invention has at least the following beneficial technical effects:
the invention can efficiently and simultaneously acquire entity representation no matter how many different entities are in the current document by representing each entity and encoding the relative positions among the entities, and further inputs the relationship results of all potential entity pairs in the document once. In addition, the present invention employs a bi-directional attention mechanism for the problem of logical reasoning between non-target entities, i.e. first obtaining each sentence and the coded representation of each entity, then for each entity, traversing the obtaining so whether the entity has a relationship to the combination and, if so, which relationship in a given set of relationships.
The developer can efficiently and accurately extract the chapter-level relation by using the method disclosed by the invention, and simultaneously solves two main problems of chapter-level relation extraction, namely the problem of calculation cost caused by traversing all entities to generate alternative samples and the problem of logic reasoning between target entities and non-target entities.
Drawings
In order to more clearly illustrate the embodiments of the invention or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flow chart of a chapter level relationship extraction method of the present invention;
FIG. 2 is a schematic diagram of a chapter level relationship extraction method of the present invention;
FIG. 3 is a schematic flow chart of the present invention for processing the document based on bi-directional attention constraints;
fig. 4 is a schematic representation of an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the technical solutions of the present invention will be described in detail below. It will be apparent that the described embodiments are only some, but not all, embodiments of the invention. All other embodiments, based on the examples herein, which are within the scope of the invention as defined by the claims, will be within the scope of the invention as defined by the claims.
Referring to fig. 1 and 2, the present invention is a chapter level relation extraction method, which includes the steps of:
s1: inputting a document to be processed, wherein the document is a chapter-level document;
s2: processing the document based on bidirectional attention constraint to obtain abstract semantic representations of entities and sentences, wherein the abstract semantic representations have global information and logical reasoning information;
s3: and judging the relation type of the target entity pair in the document based on the abstract semantic representation.
It should be noted that the bidirectional attention mechanism is based on the following intuitive observations: if a given sentence contains a large amount of information that determines what the target entity pair is, then the target entity pair, in turn, will find a closer relationship with the given sentence when calculating the relationship with each sentence in the document.
The invention mainly aims at the documents of the chapter level, namely the texts which are often hundreds or thousands in length and far larger than the sentence level, and a large number of target entities exist in the documents of the chapter level. According to the relation extraction method for the target entity in the chapter-level document based on the bidirectional attention constraint, the input chapter-level document is subjected to a series of transformation and information extraction, whether the target entity pair combination contains a relation or not is finally obtained, and if the relation is contained, the relation is which in a given relation set. The developer can efficiently and accurately extract the chapter-level relation, and simultaneously solve two main problems of chapter-level relation extraction, namely the calculation cost problem caused by traversing all entities to generating the alternative sample and the logic reasoning problem between the target entity and the non-target entity.
Referring to fig. 3, the processing the document based on the bidirectional attention constraint to obtain an abstract semantic representation of entities and sentences includes:
s21: obtaining a representation of each entity and a representation of each sentence in the document;
s22: acquiring a global entity interaction matrix based on the representation of each entity;
s23: calculating the global entity interaction matrix and the representation of each sentence based on a first attention mechanism to obtain the representation of each sentence based on entity attention;
s24: calculating the representation of each sentence based on the entity attention based on the graph convolution network to obtain the representation of each sentence with logic reasoning information;
s25: based on a second attention mechanism, calculating the global entity interaction matrix and the representation of each sentence with logical reasoning information to obtain a new global entity interaction matrix with global information and logical reasoning information, and taking the new global entity interaction matrix as the abstract semantic representation of the entity and the sentence.
In step S21, the obtaining a representation of each entity and a representation of each sentence in the document includes:
acquiring an abstract semantic representation of the document;
a representation of each entity and a representation of each sentence in the document is obtained based on the abstract semantic representation of the document.
Wherein the obtaining the abstract semantic representation of the document comprises:
performing word vector conversion on the document to obtain a word vector matrix of the document;
and carrying out bidirectional LSTM operation on the word vector matrix to obtain the abstract semantic representation of the document.
It should be noted that, the input document to be processed is encoded by using the pre-training word vector matrix, and is converted into word vectors, so as to obtain word vector representation of the document, and abstract semantic representation of the document is further obtained through the bidirectional LSTM.
Suppose there are k different entities in the document than there are n words, m sentences. Let the i-th word be denoted as w i There is a pre-trained word vector matrix of size N x d, where N is the number of words in the word vector matrix and d is the dimension of each word in the word vector matrix. Converting each word into a word vector using a pre-trained word vector matrix pair, i.e., for w i And finding out the corresponding word vectors, and splicing word vectors of all words in the document to obtain word vector representation of the whole document, wherein the size of the word vector representation is n multiplied by d.
Assuming that the number of hidden units in the bidirectional LSTM is H, after passing through the bidirectional LSTM, the abstract semantics of the document are represented as H, and the size is n×2h.
The obtaining a representation of each sentence in the document based on the abstract semantic representation of the document includes:
and extracting the composition of each sentence from the abstract semantic representation of the document according to the starting position and the ending position of each sentence, carrying out maximum pooling processing on the composition of each sentence, and taking a sentence vector obtained after the maximum pooling processing as the representation of each sentence.
It should be noted that, when the starting position and the ending position of each sentence are preset in the article to be processed, each sentence in the document is easily extracted, and the vector representation of each sentence is obtained by using the maximum pooling operation.
Assuming that a certain sentence in a document starts and ends with the a-th word and the b-th word, respectively, the sentence is expressed as:
wherein,the size of (2) is 1×2h.
The obtaining a representation of each entity in the document based on the abstract semantic representation of the document includes:
and determining words forming the entities according to the positions of each entity, calculating an average word vector of the words, taking the average word vector as an entity vector, and taking the average vector of the entity vectors of the same entity as the representation of each entity.
It should be noted that, assuming that a given target entity appears 3 times, the entity expressions are e respectively 1 1 ,e 2 1 And e 3 1 Then the representation of the current entity The size of (2) is 1×2h.
In step S22, the obtaining a global entity interaction matrix based on the representation of each entity includes:
information interaction is carried out on every two entities, and two entity vectors are converted into one entity interaction vector;
and forming a global entity interaction matrix by all entity interaction vectors.
It should be noted that, traversing all entity pairs in the current documentSimultaneously considering the sequence of the two entities (namely the directions of the entities), and then carrying out information interaction on the two entities, wherein the interaction method is to transform two entity vectors into an advanced entity interaction vector v through a bilinear layer ij ：
Assuming that the two entity vectors areThe operation of the bilinear layer is v ij ＝σ(e i W v e j +b v ) Wherein W is v ,b v Is a trainable parameter.
All entity interaction vectors are constructed as a global entity interaction matrix. Assuming that there are k entities in the current document, there are k (k-1) interaction vectors in the constructed global entity interaction matrix.
Step S23: calculating the global entity interaction matrix and the representation of each sentence based on a first attention mechanism to obtain the representation of each sentence based on entity attention; the method comprises the following specific steps:
the attention mechanism operation is performed on each sentence representation and the global entity interaction matrix, so that the invention obtains a certain logic reasoning capability, and the expression of each sentence based on the entity attention is obtained.
Specifically, from the foregoing, it can be known that a sentence is expressed asIf the representation of a certain entity interaction vector is v, the attention weight of the current sentence and the entity interaction vector is calculated as +.>Calculating the current sentence and each entity interaction vector to obtain k (k-1) attention weights, and then weighting all entity interaction vectors to obtain the representation of each sentence based on the attention of the entity interaction vector>I.e. < ->All sentences are combined into a feature matrix X by columns.
Step S24: calculating the representation of each sentence based on the entity attention based on the graph convolution network to obtain the representation of each sentence with logic reasoning information; the method comprises the following specific steps:
and carrying out graph convolution operation on the feature matrix X of the sentence through the following transformation: l=ρ (AXW) 2 ) Wherein ρ is a nonlinear activation function, e.g. sigmoid function, W 2 Is a trainable parameter. The L thus obtained is the same size as X, each column of which is a representation L of a sentence.
It should be noted that, through a graph convolution neural network layer, the sequential connection information and the logical reasoning information between each sentence are obtained. After passing this layer, each sentence representation with sequential contact information and logical inference information will be obtained.
Step S25: based on a second attention mechanism, calculating the global entity interaction matrix and the representation of each sentence with logical reasoning information to obtain a new global entity interaction matrix with global information and logical reasoning information, and taking the new global entity interaction matrix as the abstract semantic representation of the entity and the sentence. The method specifically comprises the following steps:
specifically, the step S24 can learn that a sentence is expressed asIf the representation of a certain entity interaction vector is v, the attention weight of the current entity interaction vector and sentence is calculated as +.>Calculating the current entity interaction vector and each sentence to obtain m attention weights, and then weighting all sentence vectors to obtain each entity interaction vector based on the attention of the sentenceCharacterization->I.e. < ->
It should be noted that, the matrix weights adopted by the first attention mechanism and the second attention mechanism are the same. This is because, given two target entities and contexts, the degree of association between the target entity and context should be consistent with the degree of association between the context and target entity, so that the resulting matrix for both attention mechanisms is constrained to be as close as possible. Through bi-directional attention constraints, a representation of each entity for each sentence is derived, and then a new global entity interaction matrix with global information and logical reasoning information is obtained, which is used for classification of relation extraction.
In step S3, determining a relationship type of the target entity pair in the document based on the abstract semantic representation includes:
performing column-wise weighting operation on the new global entity interaction matrix to obtain a global entity interaction vector;
and inputting the global entity interaction vector into a preset classification function for distinguishing the relationship types of the entity pairs to obtain a classification result, and taking the classification result as the relationship type of the target entity pair in the document.
Preferably, the classification function employs a softmax function.
For each entity interaction vector generated from step S25Determining which relationship it belongs to, using the softmax function:
taking the obtained roughThe relationship type corresponding to y with the maximum value is used as the entity interaction vectorIs a relationship of (3).
It should be noted that, assuming that there are predefined 3 relationships, after a global entity interaction vector passes through softmax, 3 probability values are obtained, and respectively corresponding to the probability values of the 3 relationships, which probability is the largest is predicted as which relationship. For example, the target entities of the document are: the method comprises the steps of carrying out classification function operation on Beijing capital airport, beijing and Beijing subway airport lines, wherein the obtained relationship is belongings, capital and roads after the target entity pair is subjected to the classification function operation, and the probability that the relationship corresponds to 0.7, 0.2 and 0.1 respectively is that the relationship of the target entity pair is belonged to.
The implementation is illustrated with respect to fig. 4 as follows:
1. firstly, acquiring representations of all entities (such as Beijing capital airport, beijing subway airport line and the like) and sentences in an input document;
2. acquiring an attention weight matrix based on given target entities and sentences, and new entity and sentence representations;
3. after the new sentence representation passes through the graph convolution neural network, the sentence representation with logic reasoning information is obtained, and then the second attention mechanism calculation is carried out with the entity representation;
4. constraint is carried out on the obtained calculation results of the two attentiveness mechanisms, and interaction information of each entity pair is calculated at the same time;
5. the interaction information for each entity pair is used to determine whether or not a given target entity pair has a relationship, and if so, what.
The foregoing is merely illustrative of the present invention, and the present invention is not limited thereto, and any person skilled in the art will readily recognize that variations or substitutions are within the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (9)
1. A chapter-level relationship extraction method, comprising:
inputting a document to be processed, wherein the document is a chapter-level document;
processing the document based on bidirectional attention constraint to obtain abstract semantic representations of entities and sentences, wherein the abstract semantic representations have global information and logical reasoning information;
judging the relation type of a target entity pair in the document based on the abstract semantic representation;
the processing the document based on the bidirectional attention constraint to obtain abstract semantic representation of entities and sentences comprises the following steps:
obtaining a representation of each entity and a representation of each sentence in the document;
acquiring a global entity interaction matrix based on the representation of each entity;
calculating the global entity interaction matrix and the representation of each sentence based on a first attention mechanism to obtain the representation of each sentence based on entity attention;
calculating the representation of each sentence based on the entity attention based on the graph convolution network to obtain the representation of each sentence with logic reasoning information;
based on a second attention mechanism, calculating the global entity interaction matrix and the representation of each sentence with logical reasoning information to obtain a new global entity interaction matrix with global information and logical reasoning information, and taking the new global entity interaction matrix as the abstract semantic representation of the entity and the sentence.
2. The extraction method according to claim 1, wherein the obtaining the representation of each entity and the representation of each sentence in the document comprises:
acquiring an abstract semantic representation of the document;
a representation of each entity and a representation of each sentence in the document is obtained based on the abstract semantic representation of the document.
3. The extraction method according to claim 2, wherein the obtaining a representation of each sentence in the document based on the abstract semantic representation of the document comprises:
and extracting the composition of each sentence from the abstract semantic representation of the document according to the starting position and the ending position of each sentence, carrying out maximum pooling processing on the composition of each sentence, and taking a sentence vector obtained after the maximum pooling processing as the representation of each sentence.
4. The extraction method according to claim 2, wherein the obtaining a representation of each entity in the document based on the abstract semantic representation of the document comprises:
and determining words forming the entities according to the positions of each entity, calculating an average word vector of the words, taking the average word vector as an entity vector, and taking the average vector of the entity vectors of the same entity as the representation of each entity.
5. The extraction method according to claim 2, wherein said obtaining an abstract semantic representation of said document comprises:
performing word vector conversion on the document to obtain a word vector matrix of the document;
and carrying out bidirectional LSTM operation on the word vector matrix to obtain the abstract semantic representation of the document.
6. The extraction method according to claim 4, wherein the obtaining a global entity interaction matrix based on the representation of each entity comprises:
information interaction is carried out on every two entities, and two entity vectors are converted into one entity interaction vector;
and forming a global entity interaction matrix by all entity interaction vectors.
7. The extraction method according to claim 1, wherein the matrix weights used by the first and second attention mechanisms are the same.
8. The extraction method according to claim 1, wherein said determining a relationship type of a target entity pair in the document based on the abstract semantic representation comprises:
performing column-wise weighting operation on the new global entity interaction matrix to obtain a global entity interaction vector;
and inputting the global entity interaction vector into a preset classification function for distinguishing the relationship types of the entity pairs to obtain a classification result, and taking the classification result as the relationship type of the target entity pair in the document.
9. The extraction method according to claim 8, wherein the classification function employs a softmax function.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010644404.XA CN111831783B (en) | 2020-07-07 | 2020-07-07 | Method for extracting chapter-level relation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010644404.XA CN111831783B (en) | 2020-07-07 | 2020-07-07 | Method for extracting chapter-level relation |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111831783A CN111831783A (en) | 2020-10-27 |
| CN111831783B true CN111831783B (en) | 2023-12-08 |
Family
ID=72900251
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010644404.XA Active CN111831783B (en) | 2020-07-07 | 2020-07-07 | Method for extracting chapter-level relation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111831783B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113435190B (en) * | 2021-05-18 | 2022-08-05 | 北京理工大学 | Chapter relation extraction method integrating multilevel information extraction and noise reduction |
| CN113297838A (en) * | 2021-05-21 | 2021-08-24 | 华中科技大学鄂州工业技术研究院 | Relationship extraction method based on graph neural network |
| CN113468325A (en) * | 2021-06-09 | 2021-10-01 | 广西电网有限责任公司 | Document level relation extraction method based on associated sentence selection and relation graph reasoning |
| CN113822019B (en) * | 2021-09-22 | 2024-07-12 | 科大讯飞股份有限公司 | Text normalization method, related device and readable storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106354710A (en) * | 2016-08-18 | 2017-01-25 | 清华大学 | Neural network relation extracting method |
| CN108280062A (en) * | 2018-01-19 | 2018-07-13 | 北京邮电大学 | Entity based on deep learning and entity-relationship recognition method and device |
| CN111008529A (en) * | 2019-07-24 | 2020-04-14 | 贵州大学 | Chinese relation extraction method based on neural network |
| CN111274800A (en) * | 2020-01-19 | 2020-06-12 | 浙江大学 | Inference type reading understanding method based on relational graph convolution network |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108280112B (en) * | 2017-06-22 | 2021-05-28 | 腾讯科技（深圳）有限公司 | Abstract generation method and device and computer equipment |
-
2020
- 2020-07-07 CN CN202010644404.XA patent/CN111831783B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106354710A (en) * | 2016-08-18 | 2017-01-25 | 清华大学 | Neural network relation extracting method |
| CN108280062A (en) * | 2018-01-19 | 2018-07-13 | 北京邮电大学 | Entity based on deep learning and entity-relationship recognition method and device |
| CN111008529A (en) * | 2019-07-24 | 2020-04-14 | 贵州大学 | Chinese relation extraction method based on neural network |
| CN111274800A (en) * | 2020-01-19 | 2020-06-12 | 浙江大学 | Inference type reading understanding method based on relational graph convolution network |
Non-Patent Citations (8)
| Title |
|---|
| Exploiting Entity BIO Tag Embeddings and Multi-task Learning for Relation Extraction with Imbalanced Data;Ye, Wei* 等;computer Science;1-10 * |
| Improve relation extraction with dual attention-guided graph convolutional networks;Zhixin Li 等;Neural Computing and Applications;1773–1784 * |
| 图文知识图谱中的关系抽取算法研究;李智超;中国优秀硕士学位论文全文数据库 信息科技辑(第11期);I138-572 * |
| 基于BiLSTM并结合自注意力机制和句法信息的隐式篇章关系分类;凡子威 等;计算机科学(第05期);214-220 * |
| 基于双向LSTM和自注意力机制的中文关系抽取研究;刘鉴 等;山西大学学报(自然科学版)(第01期);8-13 * |
| 基于联合注意力机制的篇章级机器翻译;李京谕 等;中文信息学报(第12期);45-53 * |
| 基于语境交互感知和模式筛选的隐式篇章关系识别;郭凤羽 等;计算机学报(第05期);901-915 * |
| 结合实体共现信息与句子语义特征的关系抽取方法;马语丹 等;中国科学:信息科学(第11期);1533-1545 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111831783A (en) | 2020-10-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109284506B (en) | User comment emotion analysis system and method based on attention convolution neural network | |
| CN111831783B (en) | Method for extracting chapter-level relation | |
| CN109033068B (en) | Method and device for reading and understanding based on attention mechanism and electronic equipment | |
| Mo et al. | Large language model (llm) ai text generation detection based on transformer deep learning algorithm | |
| CN111783394B (en) | Training method of event extraction model, event extraction method, system and equipment | |
| CN106502985B (en) | neural network modeling method and device for generating titles | |
| CN110287323B (en) | Target-oriented emotion classification method | |
| CN113987187B (en) | Public opinion text classification method, system, terminal and medium based on multi-label embedding | |
| CN113553412B (en) | Question-answering processing method, question-answering processing device, electronic equipment and storage medium | |
| Mukherjee et al. | Utilization of oversampling for multiclass sentiment analysis on amazon review dataset | |
| WO2023159767A1 (en) | Target word detection method and apparatus, electronic device and storage medium | |
| CN111145914B (en) | Method and device for determining text entity of lung cancer clinical disease seed bank | |
| Zhang et al. | A BERT fine-tuning model for targeted sentiment analysis of Chinese online course reviews | |
| CN111597815A (en) | Multi-embedded named entity identification method, device, equipment and storage medium | |
| CN111859979A (en) | Ironic text collaborative recognition method, ironic text collaborative recognition device, ironic text collaborative recognition equipment and computer readable medium | |
| CN111400494A (en) | Sentiment analysis method based on GCN-Attention | |
| CN111597816A (en) | Self-attention named entity recognition method, device, equipment and storage medium | |
| CN114064852A (en) | Method and device for extracting relation of natural language, electronic equipment and storage medium | |
| CN116644760A (en) | Dialogue text emotion analysis method based on Bert model and double-channel model | |
| Wang et al. | Deep learning sentiment classification based on weak tagging information | |
| CN114492661B (en) | Text data classification method and device, computer equipment and storage medium | |
| CN116595023A (en) | Address information updating method and device, electronic equipment and storage medium | |
| Touati-Hamad et al. | Arabic quran verses authentication using deep learning and word embeddings | |
| CN114356990A (en) | Base named entity recognition system and method based on transfer learning | |
| CN113139057A (en) | Domain-adaptive chemical potential safety hazard short text classification method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |