CN115130663B - Heterogeneous network attribute completion method based on graph neural network and attention mechanism - Google Patents
Heterogeneous network attribute completion method based on graph neural network and attention mechanism Download PDFInfo
- Publication number
- CN115130663B CN115130663B CN202211043710.3A CN202211043710A CN115130663B CN 115130663 B CN115130663 B CN 115130663B CN 202211043710 A CN202211043710 A CN 202211043710A CN 115130663 B CN115130663 B CN 115130663B
- Authority
- CN
- China
- Prior art keywords
- attribute
- node
- target node
- nodes
- network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000013528 artificial neural network Methods 0.000 title claims abstract description 42
- 238000000034 method Methods 0.000 title claims abstract description 39
- 230000007246 mechanism Effects 0.000 title claims abstract description 28
- 238000012512 characterization method Methods 0.000 claims abstract description 56
- 230000006870 function Effects 0.000 claims abstract description 5
- 238000012795 verification Methods 0.000 claims abstract description 5
- 239000011159 matrix material Substances 0.000 claims description 28
- 238000004364 calculation method Methods 0.000 claims description 21
- 238000005295 random walk Methods 0.000 claims description 18
- 239000013598 vector Substances 0.000 claims description 11
- 238000012549 training Methods 0.000 claims description 8
- 230000004931 aggregating effect Effects 0.000 claims description 7
- 230000008859 change Effects 0.000 claims description 6
- 238000010586 diagram Methods 0.000 claims description 6
- 230000004927 fusion Effects 0.000 claims description 6
- 230000009466 transformation Effects 0.000 claims description 6
- 238000004422 calculation algorithm Methods 0.000 claims description 4
- 238000010276 construction Methods 0.000 claims description 4
- 238000007781 pre-processing Methods 0.000 claims description 4
- 230000002776 aggregation Effects 0.000 claims description 3
- 238000004220 aggregation Methods 0.000 claims description 3
- 239000012633 leachable Substances 0.000 claims description 3
- 238000005259 measurement Methods 0.000 claims description 3
- 238000010606 normalization Methods 0.000 claims description 3
- 238000011176 pooling Methods 0.000 claims description 3
- 238000012360 testing method Methods 0.000 claims description 2
- 230000000295 complement effect Effects 0.000 abstract description 6
- 238000004458 analytical method Methods 0.000 abstract description 2
- 238000012545 processing Methods 0.000 abstract description 2
- 230000003044 adaptive effect Effects 0.000 description 2
- 238000007418 data mining Methods 0.000 description 2
- 238000002474 experimental method Methods 0.000 description 2
- 238000007476 Maximum Likelihood Methods 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000000052 comparative effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000009499 grossing Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000005065 mining Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/082—Learning methods modifying the architecture, e.g. adding, deleting or silencing nodes or connections
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Molecular Biology (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Devices For Executing Special Programs (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention discloses a heterogeneous network attribute completion method based on a graph neural network and an attention mechanism, and belongs to the technical field of data processing. Firstly, capturing nodes similar to target nodes (nodes with missing attributes) from a network by combining the cosine similarity between K-Nearest Neighbor and the attributes, and adaptively converting the attributes of the nodes into network characterization of the feature domain of the target nodes; based on the attention mechanism of the graph neural network and the transducer, carrying out hierarchical analysis on the topological structure in the network and the attribute information of the nodes, and further obtaining the network representation of the target node in the spatial domain; finally, combining network characterization of the spatial domain and the characteristic domain, and carrying out model parameter learning based on a loss function of Euclidean distance so as to complement the missing attribute. Through practical verification, the attribute completion method provided by the invention has the characteristics of high efficiency and high accuracy.
Description
Technical Field
The invention relates to a heterogeneous network attribute completion method based on a graph neural network and an attention mechanism, and belongs to the technical field of data processing.
Background
Networks are ubiquitous in our real life, and most of the connections between objects in the real world can be represented as networks, for example, friend relationships between users can be regarded as social networks, reference relationships between papers can be regarded as reference networks, and connection relationships between road segments can be regarded as a traffic network. The above networks are all made up of nodes of the same type, so they are also called homogeneous networks. More widely present in the real world is a heterogeneous network, the nodes constituting the heterogeneous network being of different types, for example: shopping network is composed of users and commodities, and academic network is composed of authors, the units where authors are located and papers. Although these networks contain huge amounts of data, the lack of attributes in the networks (e.g., in shopping networks, some users are reluctant to upload their age information, in academic networks, keywords of papers are not filled in completely by authors) also presents a great challenge to the exploitation of the potential value contained in the networks. The data mining method for the network can effectively improve the efficiency of data mining of the network by complementing the missing attribute in the network, but the method is complex, and the considered factors are many, such as the connection relationship between network nodes, and the relationship between the existing attribute and the missing attribute of the network nodes. How to efficiently and accurately complement missing data in a network is also becoming increasingly important to academia and industry.
In the field of attribute completion, a conventional attribute completion method generally performs analysis completion from connection relationships of nodes in a network and semantic text information. The above method does not take into account the topology of the entire network. In recent years, the graph neural network method shows higher efficiency and accuracy in mining network information, such as graph convolution neural network and graph annotation force network show excellent performance in capturing network topology structure and node attribute information. The development of the graph neural network also brings new possibilities to the field of attribute completion, such as a method for carrying out joint learning based on the graph convolution neural network and combining attribute completion and commodity recommendation, and attribute completion is carried out by using the representation learned by the graph neural network. These methods have achieved significant results in attribute completion, but there is still room for improvement.
By analyzing and summarizing the existing attribute completion method, the existing method has the following defects: 1) The network characterization in the heterogeneous network is different from that of all nodes in the homogeneous network, and the types of all nodes in the heterogeneous network are the same, so that the missing attribute of the nodes in the heterogeneous network is complemented, and the homogeneous node information and the heterogeneous node information need to be comprehensively considered. This problem has plagued many network characterization models. 2) The graph neural network cannot efficiently capture high-order node information in the network. GCN essentially generates a characterization by aggregating the surrounding node information of the target node under a semi-supervised framework. In the attribute completion problem, the attribute of the target node may be related to not only the surrounding nodes but also the higher-order nodes. If the higher-order information of the target node can be captured through stacking the layer number of GCNs, the node-to-node characterization in the network is more and more similar with the increase of the layer number of GCNs, the specificity is lost, and the accuracy of attribute complementation is further affected.
Disclosure of Invention
In order to solve the heterogeneous network attribute completion problem more effectively, the invention aims to provide an attribute completion method based on a graph neural network and an attention mechanism so as to further improve the efficiency and accuracy of attribute completion and provide a method and technical support for the heterogeneous network attribute completion problem.
In order to achieve the aim of the invention, the invention adopts the following specific technical scheme:
an attribute completion method based on a graph neural network and an attention mechanism comprises the following steps:
s1: acquiring a heterogeneous attribute network with missing attributes, wherein a node with missing attributes is called a target node, and a node with complete attributes is called a source node;
s2: k source nodes most similar to the existing attribute of the target node are selected by adopting a K nearest neighbor (K-nearest neighbor) algorithm to complete the attribute in the feature space;
s3: after K source nodes which are most similar to the target node are screened out by utilizing cosine similarity, a learnable parameter is endowed to each source node to dynamically adjust the weight of the attribute of each source node for complementing the attribute of the target node, and attribute characterization endowed with weight of the feature space is obtained;
s4: the nodes directly connected with the target node in the heterogeneous attribute network are aggregated with the target node to obtain a low-order representation of the structural space; the specific aggregation mode is realized by simplifying a graph neural network (Simplifying graph convolutional networks);
s5: firstly, obtaining a high-order node of a target node by using a random walk mode;
s6: different weights are given to nodes in the high-order node sequence based on a transducer, and high-order characterization of the given weights of the structural space is obtained;
s7: splicing the attribute representation of the characteristic space, the low-order representation of the structural space and the high-order representation of the structural space, and then sending the spliced representation into a multi-layer perceptron to convert the representation into the attribute of the target node;
s8: the method comprises the steps of manually removing part of attributes of part of nodes by adopting a supervised learning mode, then complementing the attributes by reconstruction prediction, continuously training through the difference between the complemented attributes and the true attributes, and finally complementing the attribute missing values of other nodes by utilizing a trained model.
Further, in the step S1, the heterogeneous network with the missing attribute is defined asRepresenting the set of vertices in the graph, +.>Representing the set of edges in the diagram, +.>Attribute matrix representing vertices in the graph, +.>For the number of vertices in the graph, < > and->For each dimension of vertex feature, +.>Is a marking matrix, when->In the time-course of which the first and second contact surfaces,the corresponding attribute in (a) is missing; on the contrary, let(s)>The corresponding attributes of (a) are complete.
Further, in the S2, a large number of nodes are typically included in the heterogeneous network, and it is inefficient and impractical to complement the missing attribute of the target node with the attributes of all the source nodes; therefore, K nearest neighbors are adopted to select K source nodes which are most similar to the existing attribute of the target node to complete the attribute in the feature space, and cosine similarity (formula (1)) is used to measure the similarity of two attribute vectors;
(1)
wherein ,representing the similarity of two nodes, +.>Representing the existing properties of the target node +.>Representing an attribute in the source node corresponding to the existing attribute of the target node,/->The larger the value, the higher the similarity between the target node and the source node.
Further, in the S3, there may be a plurality of factors affecting the attribute complement relationship of the source node to the target node in the heterogeneous network, for example: whether nodes are directly connected with each other, weights on the connected node edges, and the like. The dynamic adjustment is performed according to formula (2):
（2）
wherein ,for the characterization of the feature space of the target node, +.>Is->Is most similar->Set of individual source nodes->A leachable adjustment weight for each source node, +.>Is a feature vector of the source node.
Further, in the step S4, after the feature space finishes the feature learning of the target node, the feature needs to be learned on the target node in the structural space; the specific formula (3) is as follows:
（3）
wherein ,to simplify the drawing neural network +.>Weight matrix of layer,/>Is->Layer output, in the first layer of the reduced graph neural network +.>Is an adjacency matrix.
The graph neural network essentially captures topological structure information and attribute information of nodes in the network by aggregating neighbor attributes of the target nodes, but as the layers of the graph neural network are stacked, the characteristic aggregated by the target nodes is over-smoothed, that is to say, the characteristics obtained by aggregating the characteristic of the target nodes lose specificity and distinguishing. Therefore, a simplified graph neural network is used in the structure space for aggregating neighbor nodes of the target node, and only one layer of network is used, and the specificity of the characterization of the target node is reserved while the topological structure and the attribute characteristics are captured.
Further, the step S5 is specifically: in order to capture high-order information of a target node in a heterogeneous network, the high-order node of the target node is obtained by using a random walk mode, wherein the random walk is a mode of traversing the node in the graph combining the advantages of depth-first traversal and breadth-first traversal in the graph, and the specific traversing mode is shown in a formula (4):
（4）
wherein Representing node->To node->Is the weight of the edge of (2);
for each target node, taking the target node as a root node, and carrying out random walk based on the root node to acquire a node sequence. Because directly connected source nodes have been used in the reduced graph neural network, these nodes are deleted in the random walk node sequence.
Further, in S6, the influence of the node sequence generated by the random walk on the target node is different, so the nodes in the node sequence are given different weights based on the transformers. The method comprises the following steps:
s6-1: firstly, carrying out linear change on the characteristics of a target node and a source node sequence, then calculating the weights of the target node and the nodes in each sequence based on the linear change, and carrying out softmax normalization operation on the obtained weights for the stability of calculation;
s6-2: then, the node features in the source node sequence are again subjected to a new linear transformation independent of the weight calculation.
S6-3: and finally, giving weight to the node characteristics of the source node sequence after each linear transformation, and accumulating the node characteristics to obtain the node characteristics of the target node higher order, wherein the calculation method is shown in a formula (5):
（5）
wherein Representing node->Characterization aggregated by a simplified graph neural network, +.>Representing node->Is characterized by (1)>，/> and />Projection parameter matrix representing a parameter which can be learned, +.>Representing node->Higher order network characterization of (2).
S6-4: on this basis, the attention mechanism is extended to a multi-headed attention mechanism to capture multiple dependency relationships between the target node and the Gao Jieyuan node. Then, the multiple dependency relationships are sent to an average pooling layer to obtain final high-order network characterization, and the calculation method is shown in a formula (6):
（6）
wherein ,representing node->In->Obtained higher order network characterization in the sub-attentive mechanism,/->Representing the total number of times that the attention mechanism calculation needs to be made.
Further, in S7, a specific calculation manner of the splicing is shown in formula (7):
(7)
wherein Representing vector concatenation operations,/->For the predicted target node->Is a property value of (a).
Further, the step S8 specifically includes:
s8-1: after the predicted attribute of the target node is obtained, the original existing attribute is reserved, the predicted attribute is filled in the missing attribute to complete the task of attribute completion, and the filling method is shown in a formula (8):
（8）
wherein ,representing predicted attributes of all nodes after attribute completion, +.>Representing Hadamard product, ->A matrix with all 1 elements:
s8-2: the difference between the predicted attribute and the real attribute of the loss function measurement is set based on the Euclidean distance formula, and the specific calculation mode is as shown in formula (9):
（9）
wherein Representing the set of target nodes>For node->True missing attribute value, +_>Is a nodePredicted missing attribute values.
The invention considers the relevance between the target node and the source node in the attribute space and the relevance in the structural space simultaneously when the heterogeneous network attribute is fully filled, and is specifically expressed as follows: in attribute space useThe neighbor algorithm finds the most similar +.>The source node then adaptively gives it the corresponding +.>And adding the weight values to obtain the attribute space characterization of the target node. In the structural space, a simplified graph neural network is used for aggregating first-order neighbor information of the target node to obtain low-order representation of the target node; a multi-headed attention mechanism based on a transducer and random walk is then used to capture high-order neighbor information of the target node in the structural space to obtain a high-order representation of the target node. And finally, fusing the three characterizations, updating parameters of the whole model under the guidance of a loss function based on Euclidean distance, and finally complementing the missing attribute of the target node.
The invention has the advantages and beneficial effects that:
compared with the traditional attribute completion method, the method introduces a framework for network characterization learning to perform attribute completion, and can better capture the structural characteristics of the target node in the network; the first-order neighbors of the target node are aggregated by using the simplified graph neural network, so that the accuracy of the obtained network representation can be ensured on the premise of improving the attribute completion time efficiency; the method uses a random walk and transformer-based attention mechanism to capture the high-order neighbor information of the target node, and solves the problem of node characteristic smoothing caused by simple stacking of graph neural network modules to a certain extent.
Through practical verification, the attribute completion method provided by the invention has the characteristics of high efficiency and high accuracy.
Drawings
Fig. 1 is an overall flow chart of the present invention.
Fig. 2 is a frame diagram of the present invention.
Fig. 3 is a first topological structure diagram of weights between nodes obtained in the present invention.
Fig. 4 is a second topology diagram of the weights between nodes obtained in the present invention.
FIG. 5 is a flow chart of the present invention for obtaining higher order node characterizations based on an attention mechanism.
Detailed Description
The invention will be further described with reference to fig. 1-5 and the accompanying examples.
Example 1:
an attribute completion method based on a graph neural network and an attention mechanism is shown in a whole flow chart in figure 1. The method comprises the following steps:
s1: acquiring a heterogeneous attribute network with missing attributes, wherein a node with missing attributes is called a target node, and a node with complete attributes is called a source node; defining heterogeneous property networks with missing properties asRepresenting the set of vertices in the graph +.> Representing the set of edges in the diagram, +.>Attribute matrix representing vertexes in the graph, n is the number of vertexes in the graph, +.>For each dimension of vertex feature, +.>Is a marking matrix if +.>Then indicate +.>The corresponding attribute in (a) is missing; otherwise, it means ++>The corresponding attributes in (a) are complete;
s2: k source nodes most similar to the existing attribute of the target node are selected by adopting a K nearest neighbor (K-nearest neighbor) algorithm to complete the attribute in the feature space; the typically inclusion of a large number of nodes in a heterogeneous network, it is inefficient and impractical to complement the missing attributes of the target node with the attributes of all source nodes; therefore, K nearest neighbors are adopted to select K source nodes which are most similar to the existing attribute of the target node to complete the attribute in the feature space, and cosine similarity (formula (1)) is used to measure the similarity of two attribute vectors;
(1)
wherein ,representing the similarity of two nodes, +.>Representing the existing properties of the target node +.>Representing an attribute in the source node corresponding to the existing attribute of the target node,/->The larger the value is, the higher the similarity between the target node and the source node is;
s3: after K source nodes which are most similar to the target node are screened out by utilizing cosine similarity, a learnable parameter is endowed to each source node to dynamically adjust the weight of the attribute of each source node for complementing the attribute of the target node, and attribute characterization endowed with weight of the feature space is obtained; there may be a number of factors in the heterogeneous network that affect the attribute completion relationship of the source node to the target node, such as: whether nodes are directly connected with each other, weights on the connected node edges, and the like. The dynamic adjustment is performed according to formula (2):
（2）
wherein ,for the characterization of the feature space of the target node, +.>Is->Is most similar->Set of individual source nodes->A leachable adjustment weight for each source node, +.>Is a feature vector of the source node;
s4: the nodes directly connected with the target node in the heterogeneous attribute network are aggregated with the target node to obtain a low-order representation of the structural space; the specific aggregation mode is realized by simplifying a graph neural network (Simplifying graph convolutional networks); after the feature space finishes the representation learning of the target node, the target node needs to be learned and characterized in the structural space; the specific formula (3) is as follows:
（3）
wherein ,to simplify the drawing neural network +.>Weight matrix of layer,/>Is->Layer output, in the first layer of the reduced graph neural network +.>Is an adjacency matrix;
s5: firstly, obtaining a high-order node of a target node by using a random walk mode; in order to capture high-order information of a target node in a heterogeneous network, the high-order node of the target node is obtained by using a random walk mode, wherein the random walk is a mode of traversing the node in the graph combining the advantages of depth-first traversal and breadth-first traversal in the graph, and the specific traversing mode is shown in a formula (4):
（4）
wherein Representing node->To node->Is the weight of the edge of (2);
for each target node, taking the target node as a root node, and carrying out random walk based on the root node to acquire a node sequence. Because directly connected source nodes have been used in the reduced graph neural network, these nodes are deleted in the random walk node sequence;
s6: different weights are given to nodes in the high-order node sequence based on a transducer, and high-order characterization of the given weights of the structural space is obtained; the method comprises the following steps:
s6-1: firstly, carrying out linear change on the characteristics of a target node and a source node sequence, then calculating the weights of the target node and the nodes in each sequence based on the linear change, and carrying out softmax normalization operation on the obtained weights for the stability of calculation;
s6-2: then, the node features in the source node sequence are again subjected to a new linear transformation independent of the weight calculation.
S6-3: and finally, giving weight to the node characteristics of the source node sequence after each linear transformation, and accumulating the node characteristics to obtain the node characteristics of the target node higher order, wherein the calculation method is shown in a formula (5):
（5）
wherein Representing node->Characterization aggregated by a simplified graph neural network, +.>Representing node->Is characterized by (1)>，/> and />Projection parameter matrix representing a parameter which can be learned, +.>Representing node->Higher order network characterization of (2).
S6-4: on this basis, the attention mechanism is extended to a multi-headed attention mechanism to capture multiple dependency relationships between the target node and the Gao Jieyuan node. Then, the multiple dependency relationships are sent to an average pooling layer to obtain final high-order network characterization, and the calculation method is shown in a formula (6):
（6）
wherein ,representing node->In->Obtained higher order network characterization in the sub-attentive mechanism,/->Representing the number of times a total need for attention mechanism calculations;
s7: splicing the attribute representation of the characteristic space, the low-order representation of the structural space and the high-order representation of the structural space, and then sending the spliced representation into a multi-layer perceptron to convert the representation into the attribute of the target node; the specific calculation mode of the splicing is shown in a formula (7):
（7）
wherein Representing vector concatenation operations,/->For the predicted target node->Is a property value of (a).
S8: the method comprises the steps of adopting a supervised learning mode, firstly manually removing part of attributes of part of nodes, then complementing the attributes through reconstruction prediction, continuously training through the difference between the complemented attributes and the true attributes, and finally complementing attribute missing values of other nodes by using a model which is completed through training; the method comprises the following steps:
s8-1: after the predicted attribute of the target node is obtained, the original existing attribute is reserved, the predicted attribute is filled in the missing attribute to complete the task of attribute completion, and the filling method is shown in a formula (8):
（8）
wherein ,representing predicted attributes of all nodes after attribute completion, +.>Representing Hadamard product, ->A matrix with all 1 elements;
s8-2: the difference between the predicted attribute and the real attribute of the loss function measurement is set based on the Euclidean distance formula, and the specific calculation mode is as shown in formula (9):
（9）
wherein Representing the set of target nodes>For node->True missing attribute value, +_>Is a nodePredicted missing attribute values.
Example 2: this example uses example 1 as a basic method, and a module design is performed.
The attribute completion system based on the graph neural network and the attention mechanism comprises a data preprocessing module, a marking matrix construction module, a feature space characterization learning module, a low-order neighbor characterization learning module, a high-order neighbor characterization learning module, a characterization fusion module and an attribute reasoning module, wherein the details of the parts are as follows as shown in fig. 2:
the data preprocessing module is used for: firstly normalizing attribute characteristics in an original data set, dividing the data into a training set, a test set and a verification set, randomly removing attribute information of nodes in the training set, and recording the removed attribute information as a true value to guide a model to learn.
The marking matrix construction module: traversing the node attributes in the dataset, marking the missing attribute values of the nodes, and further forming an attribute marking matrix。
The feature space representation learning module: and selecting nodes similar to the target node in the structural space, then giving weight to the nodes, and summing the characteristics of the nodes to obtain the characterization of the attribute space of the target node.
The low-order neighbor characterization learning module: and (3) aggregating the attributes of the first-order neighbor nodes of the target node by using the simplified graph neural network, and acquiring the low-order neighbor characterization of the target node as shown in fig. 3.
The high-order neighbor characterization learning module: the attribute of the high-order neighbor node of the target node is aggregated through random walk and a attention mechanism based on a transducer, and as shown in fig. 4 and 5, the high-order neighbor characterization of the target node is obtained.
The characterization fusion module: and fusing the feature space characterization, the low-order neighbor node characterization and the high-order neighbor node characterization of the target node.
The attribute reasoning module: predicting the fusion characterization of the target node through a multi-layer perceptron to obtain the predicted attribute of the target node, and filling the predicted attribute into the corresponding missing attribute through a marking matrix.
Example 3: the embodiment performs instance verification based on the method and the system
In order to verify the accuracy of model attribute completion, the invention provides the following three data sets: experiments were performed on database systems and program logic networks (DataBase systems and Logic Programming, DBLP), international computer society networks (Association for Computing Machinery, ACM) and internet movie databases (Internet Movie Database, IMDb), using Heat Kernel and corestation as evaluation metrics, and compared to seven existing models.
The seven existing models are respectively: matrix complement (Matrix Completion, MC), maximum likelihood estimation (Expectation Maximization, EM), multi-layer perceptron (Multilayer Perceptron, MLP)
) Support vector regression (Support Vector Regression, SVR), heterogeneous graph annotation force network (heterogeneous graph attention networks, HGAT), adaptive graph neural network (Adaptive Graph Convolutional Network, AGCN), and heterogeneous graph neural network by attribute-complementing (Heterogeneous Graph Neural Network via Attribute Completion, HGNN-AC).
Table 1 results of comparative experiments
The final experimental results are shown in Table 1, where AC-HEN is the method provided by the present invention. It can be seen that on three real data sets, both the Heat Kernel and the corestation of the attribute completion method provided by the invention are significantly higher than those of other methods, which means that the model constructed by the invention is superior to other existing models, and the accuracy of attribute completion is higher.
The above-mentioned plan is merely an implementation method in the present invention, but the scope of the present invention is not limited thereto, and all those skilled in the art should understand that the conceivable substitutions or alterations are included in the scope of the present invention, so the scope of the present invention shall be defined by the scope of the claims.
Claims (9)
1. The attribute completion system based on the graph neural network and the attention mechanism is characterized by comprising a data preprocessing module, a marking matrix construction module, a feature space characterization learning module, a low-order neighbor characterization learning module, a high-order neighbor characterization learning module, a characterization fusion module and an attribute reasoning module: the data preprocessing module is used for: attribute features in an original dataset are first normalized, the dataset comprising: database system and program logic network, internet computer science network and Internet film database; then dividing the data into a training set, a testing set and a verification set, randomly removing attribute information of nodes in the training set, and recording the removed attribute information as a true value to guide a model to learn; the marking matrix construction module: traversing the node attributes in the data set, marking the missing attribute values of the nodes, and further forming an attribute marking matrix M; the feature space representation learning module: selecting nodes similar to the target node in the structural space, then giving weight to the nodes, and summing the characteristics of the nodes to obtain the representation of the attribute space of the target node; the low-order neighbor characterization learning module: aggregating the attribute of the first-order neighbor node of the target node by using the simplified graph neural network to obtain the low-order neighbor characterization of the target node; the high-order neighbor characterization learning module: the attribute of a high-order neighbor node of the target node is aggregated through random walk and a transducer-based attention mechanism, and the high-order neighbor characterization of the target node is obtained; the characterization fusion module: fusing the feature space representation, the low-order neighbor node representation and the high-order neighbor node representation of the target node; the attribute reasoning module: predicting the fusion characterization of the target node through a multi-layer perceptron to obtain a predicted attribute of the target node, and filling the predicted attribute into a corresponding missing attribute through a marking matrix; the completion method of the attribute completion system based on the graph neural network and the attention mechanism comprises the following steps:
s1: acquiring a heterogeneous attribute network with missing attributes, wherein a node with missing attributes is called a target node, and a node with complete attributes is called a source node;
s2: k source nodes which are most similar to the existing attribute of the target node are selected by adopting a K neighbor algorithm to complete the attribute in the feature space;
s3: after K source nodes which are most similar to the target node are screened out by utilizing cosine similarity, a learned parameter is endowed to each source node to dynamically adjust the weight of the attribute of each source node for complementing the attribute of the target node, and attribute characterization of the characteristic space endowed with the weight is obtained;
s4: the nodes directly connected with the target node in the heterogeneous attribute network are aggregated with the target node to obtain a low-order representation of the structural space; the specific aggregation mode is realized by simplifying a graph neural network;
s5: firstly, obtaining a high-order node of a target node by using a random walk mode;
s6: different weights are given to nodes in the high-order node sequence based on a transducer, and high-order characterization of the given weights of the structural space is obtained;
s7: splicing the attribute representation of the characteristic space, the low-order representation of the structural space and the high-order representation of the structural space, and then sending the spliced representation into a multi-layer perceptron to convert the representation into the attribute of the target node;
s8: the method comprises the steps of manually removing part of attributes of part of nodes by adopting a supervised learning mode, then complementing the attributes by reconstruction prediction, continuously training through the difference between the complemented attributes and the true attributes, and finally complementing the attribute missing of other nodes by utilizing a trained model.
2. The attribute completion system according to claim 1, wherein in S1, a heterogeneous network with missing attributes is defined as Represents the set of vertices in the graph, ε represents the set of edges in the graph, +.>Attribute matrix representing vertexes in the graph, n is the number of vertexes in the graph, m is the dimension of each vertex feature,/for each vertex feature>Is a marking matrix, when->When (I)>The corresponding attribute in (a) is missing; on the contrary, let(s)>The corresponding attributes of (a) are complete.
3. The attribute completion system according to claim 1, wherein in S2, K source nodes most similar to the existing attribute of the target node are selected by K neighbors to perform attribute completion in the feature space, and cosine similarity is used to measure similarity of two attribute vectors;
wherein ,Su，v Representing the similarity of the two nodes,representing the existing properties of the target node +.>Representing the attribute corresponding to the existing attribute of the target node in the source node, S u，v The larger the value, the higher the similarity between the target node and the source node.
4. The property completion system of claim 1, wherein in S3, the dynamic adjustment is performed according to formula (2):
wherein ,for characterization of the feature space of the target node, N k (V u ) Is in contact with the target node V u Is the most similar set of k source nodes,/-)>For each source node corresponding to a leachable adjustment weight, X v Is a feature vector of the source node.
5. The attribute completion system according to claim 1, wherein in S4, after the feature space finishes the feature learning of the target node, the feature needs to be learned for the target node in the structural space; the specific formula (3) is as follows:
H (l) ＝A*H (l-1) *W (l) (3)
wherein ,W(l) To simplify the weight matrix of the first layer in the graph neural network, H (l) Is the output of the first layer, at the first layer of the reduced graph neural networkA is an adjacency matrix.
6. The attribute completion system of claim 1, wherein S5 is specifically: in order to capture high-order information of a target node in a heterogeneous network, the high-order node of the target node is obtained by using a random walk mode, wherein the random walk is a mode of a node in a traversal diagram combining the advantages of depth-first traversal and breadth-first traversal, and the specific traversal mode is shown in a formula (4):
wherein wvt Representing node v v To node v t Is a weight of an edge of (c).
7. The attribute completion system of claim 1, wherein S6 is specifically:
s6-1: firstly, carrying out linear change on the characteristics of a target node and a source node sequence, then calculating the weights of the target node and the nodes in each sequence based on the linear change, and carrying out softmax normalization operation on the obtained weights for the stability of calculation;
s6-2: then, performing new linear transformation independent of weight calculation on the node characteristics in the source node sequence again;
s6-3: finally, giving weight to the node characteristics of the source node sequence after each linear transformation, and accumulating the node characteristics to obtain the node characteristics of the target node higher order; the calculation method is shown in the formula (5):
wherein Representing node v u Characterization aggregated by a simplified graph neural network, +.>Representing node v u Is characterized by (1)> and />Projection parameter matrix representing a parameter which can be learned, +.>Representing node v u Higher order network characterization of (2);
s6-4: expanding the attention mechanism to a multi-head attention mechanism to capture multiple dependency relations between a target node and a Gao Jieyuan node; then, the multiple dependency relationships are sent to an average pooling layer to obtain final high-order network characterization, and the calculation method is shown in a formula (6):
wherein ,representing node v u The higher order network characterization obtained in the ith attention mechanism, # head represents the total number of times that the attention mechanism calculation needs to be performed.
8. The attribute completion system according to claim 1, wherein in S7, a specific calculation manner of the stitching is as shown in formula (7):
where Concat (,) represents a vector concatenation operation,for the predicted target node v u Is a property value of (a).
9. The attribute completion system of claim 1, wherein S8 is specifically:
s8-1: after the predicted attribute of the target node is obtained, the original existing attribute is reserved, the predicted attribute is filled in the missing attribute to complete the task of attribute completion, and the filling method is shown in a formula (8):
wherein ,indicating the predicted attribute of all nodes after attribute complementation, wherein, as follows, the Hadamard product is indicated, E is the matrix with the element of 1;
s8-2: the difference between the predicted attribute and the real attribute of the loss function measurement is set based on the Euclidean distance formula, and the specific calculation mode is as shown in formula (9):
wherein vtarget A set of target nodes is represented and,for node v u True missing attribute value, +_>For node v u Predicted missing attribute values.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211043710.3A CN115130663B (en) | 2022-08-30 | 2022-08-30 | Heterogeneous network attribute completion method based on graph neural network and attention mechanism |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211043710.3A CN115130663B (en) | 2022-08-30 | 2022-08-30 | Heterogeneous network attribute completion method based on graph neural network and attention mechanism |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115130663A CN115130663A (en) | 2022-09-30 |
| CN115130663B true CN115130663B (en) | 2023-10-13 |
Family
ID=83388013
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211043710.3A Active CN115130663B (en) | 2022-08-30 | 2022-08-30 | Heterogeneous network attribute completion method based on graph neural network and attention mechanism |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115130663B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115759199B (en) * | 2022-11-21 | 2023-09-26 | 山东大学 | Multi-robot environment exploration method and system based on hierarchical graph neural network |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113094484A (en) * | 2021-04-07 | 2021-07-09 | 西北工业大学 | Text visual question-answering implementation method based on heterogeneous graph neural network |
| WO2021179640A1 (en) * | 2020-03-10 | 2021-09-16 | 深圳大学 | Graph model-based short video recommendation method, intelligent terminal and storage medium |
| WO2021179838A1 (en) * | 2020-03-10 | 2021-09-16 | 支付宝(杭州)信息技术有限公司 | Prediction method and system based on heterogeneous graph neural network model |
| CN114692867A (en) * | 2022-03-24 | 2022-07-01 | 大连理工大学 | Network representation learning algorithm combining high-order structure and attention mechanism |
| CN114723037A (en) * | 2022-02-25 | 2022-07-08 | 上海理工大学 | Heterogeneous graph neural network computing method for aggregating high-order neighbor nodes |
-
2022
- 2022-08-30 CN CN202211043710.3A patent/CN115130663B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2021179640A1 (en) * | 2020-03-10 | 2021-09-16 | 深圳大学 | Graph model-based short video recommendation method, intelligent terminal and storage medium |
| WO2021179838A1 (en) * | 2020-03-10 | 2021-09-16 | 支付宝(杭州)信息技术有限公司 | Prediction method and system based on heterogeneous graph neural network model |
| CN113094484A (en) * | 2021-04-07 | 2021-07-09 | 西北工业大学 | Text visual question-answering implementation method based on heterogeneous graph neural network |
| CN114723037A (en) * | 2022-02-25 | 2022-07-08 | 上海理工大学 | Heterogeneous graph neural network computing method for aggregating high-order neighbor nodes |
| CN114692867A (en) * | 2022-03-24 | 2022-07-01 | 大连理工大学 | Network representation learning algorithm combining high-order structure and attention mechanism |
Non-Patent Citations (3)
| Title |
|---|
| "Self-training method based on GCN for semi-supervised short text classification";Hongyan Cui等;《Information Sciences》;全文 * |
| 基于注意力机制增强图卷积神经网络的个性化新闻推荐;杨宝生;;兰州文理学院学报(自然科学版)(第05期);全文 * |
| 网络表示学习算法综述;丁钰;魏浩;潘志松;刘鑫;;计算机科学(第09期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115130663A (en) | 2022-09-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113961759B (en) | Abnormality detection method based on attribute map representation learning | |
| CN109389151B (en) | Knowledge graph processing method and device based on semi-supervised embedded representation model | |
| Liu et al. | Fast attributed multiplex heterogeneous network embedding | |
| CN110442802B (en) | Multi-behavior preference prediction method for social users | |
| CN113255895A (en) | Graph neural network representation learning-based structure graph alignment method and multi-graph joint data mining method | |
| CN115130663B (en) | Heterogeneous network attribute completion method based on graph neural network and attention mechanism | |
| CN116628597A (en) | Heterogeneous graph node classification method based on relationship path attention | |
| Chen et al. | Heterogeneous graph convolutional network with local influence | |
| CN115699058A (en) | Feature interaction through edge search | |
| CN117349743A (en) | Data classification method and system of hypergraph neural network based on multi-mode data | |
| CN117556148B (en) | Personalized cross-domain recommendation method based on network data driving | |
| Zhang et al. | Graph alternate learning for robust graph neural networks in node classification | |
| Wang et al. | Community discovery algorithm of complex network attention model | |
| Pamir et al. | Electricity theft detection for energy optimization using deep learning models | |
| Sun et al. | LSEnet: Lorentz Structural Entropy Neural Network for Deep Graph Clustering | |
| CN117408336A (en) | Entity alignment method for structure and attribute attention mechanism | |
| CN117236374A (en) | Layering interpretation method based on fully developed material graph neural network | |
| CN115174421B (en) | Network fault prediction method and device based on self-supervision unwrapping hypergraph attention | |
| CN114298854B (en) | Weak supervision user identity linking method combining learning representation and alignment | |
| CN116467466A (en) | Knowledge graph-based code recommendation method, device, equipment and medium | |
| CN116976402A (en) | Training method, device, equipment and storage medium of hypergraph convolutional neural network | |
| CN113159976B (en) | Identification method for important users of microblog network | |
| Zhou et al. | A structure distinguishable graph attention network for knowledge base completion | |
| CN114611668A (en) | Vector representation learning method and system based on heterogeneous information network random walk | |
| Ghadirian et al. | Hybrid adaptive modularized tri-factor non-negative matrix factorization for community detection in complex networks |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |