research-article

Graph Convolutional Network with elastic topology

Authors:

Zhaoliang Chen,

Shiping WangAuthors Info & Claims

Volume 151, Issue C

https://doi.org/10.1016/j.patcog.2024.110364

Published: 09 July 2024 Publication History

Abstract

Graph Convolutional Network (GCN) has drawn widespread attention in data mining on graphs due to its outstanding performance and rigor theoretical guarantee. However, some recent studies have revealed that GCN-based methods may mine latent information insufficiently owing to the underutilization of the feature space. Besides, the unlearnable topology also significantly imperils the performance of GCN-based methods. In this paper, we conduct experiments to investigate these issues, finding that GCN does not fully consider the potential structure in the feature space, and a fixed topology deteriorates the robustness of GCN. Thus, it is desired to distill node features and establish a learnable graph. Motivated by this goal, we propose a framework dubbed Graph Convolutional Network with elastic topology (GCNet The source code is available at https://github.com/ZhihaoWu99/GCNet. ). With the analysis of the optimization for the proposed flexible Laplacian embedding, GCNet is naturally constructed by alternative graph convolutional layers and adaptive topology learning layers. GCNet aims to deeply explore the feature space and employ the mined information to construct a learnable topology, which leads to a more robust graph representation. In addition, a set-level orthogonal loss is utilized to meet the orthogonal constraint required by the flexible Laplacian embedding and promote better class separability. Moreover, comprehensive experiments indicate that GCNet achieves remarkable performance and generalization on several real-world datasets.

Highlights

•

Present a more flexible Laplacian embedding, which suppresses the affect of outliers.

•

Propose a block-based framework with learnable topology inspired by the flexible optimization.

•

A set-level orthogonal loss is put forward, which improves the cross-entropy loss.

•

The proposed method shows superiority via comprehensive experiments.

References

[1]

T.N. Kipf, M. Welling, Semi-Supervised Classification with Graph Convolutional Networks, in: Proceedings of the 5th International Conference on Learning Representations, 2017, pp. 1–14.

[2]

Manessi F., Rozza A., Manzo M., Dynamic graph convolutional networks, Pattern Recognit. 97 (2020).

[3]

Zhang Z., Chen C., Chang Y., Hu W., Xing X., Zhou Y., Zheng Z., SHNE: Semantics and homophily preserving network embedding, IEEE Trans. Neural Netw. Learn. Syst. (2021) 1–12.

[4]

Wu Z., Zhang Z., Fan J., Graph convolutional kernel machine versus graph convolutional networks, Adv. Neural Inf. Process. Syst. 36 (2023) 1–14.

[5]

Zhang C., Xue S., Li J., Wu J., Du B., Liu D., Chang J., Multi-aspect enhanced graph neural networks for recommendation, Neural Netw. 157 (2023) 90–102.

[6]

Dai Q., Wu X., Fan L., Li Q., Liu H., Zhang X., Wang D., Lin G., Yang K., Personalized knowledge-aware recommendation with collaborative and attentive graph convolutional networks, Pattern Recognit. 128 (2022).

[7]

Chen Y., Huang L., Wang C., Lai J., Hybrid-order gated graph neural network for session-based recommendation, IEEE Trans. Ind. Inform. 18 (3) (2022) 1458–1467.

[8]

Passos L.A., Papa J.P., Del Ser J., Hussain A., Adeel A., Multimodal audio-visual information fusion using canonical-correlated graph neural network for energy-efficient speech enhancement, Inf. Fusion 90 (2023) 1–11.

[9]

Gao Y., Feng Y., Ji S., Ji R., HGNN+: General hypergraph neural networks, IEEE Trans. Pattern Anal. Mach. Intell. 45 (3) (2023) 3181–3199.

[10]

Wu Z., Lin X., Lin Z., Chen Z., Bai Y., Wang S., Interpretable graph convolutional network for multi-view semi-supervised learning, IEEE Trans. Multimed. 25 (2023) 8593–8606.

[11]

Chen Y., Xiao X., Peng C., Lu G., Zhou Y., Low-rank tensor graph learning for multi-view subspace clustering, IEEE Trans. Circuits Syst. Video Technol. 32 (1) (2022) 92–104.

Digital Library

[12]

Cai J., Wang S., Xu C., Guo W., Unsupervised deep clustering via contractive feature representation and focal loss, Pattern Recognit. 123 (2022).

Digital Library

[13]

Yu J., Jia A.L., Mlgal: multi-level label graph adaptive learning for node clustering in the attributed graph, Knowl.-Based Syst. 278 (2023).

[14]

K. Ranasinghe, M. Naseer, M. Hayat, S.H. Khan, F.S. Khan, Orthogonal Projection Loss, in: Proceedings of IEEE/CVF International Conference on Computer Vision, 2021, pp. 12313–12323.

[15]

Q. Li, Z. Han, X. Wu, Deeper Insights Into Graph Convolutional Networks for Semi-Supervised Learning, in: Proceedings of the 32rd AAAI Conference on Artificial Intelligence, 2018, pp. 3538–3545.

[16]

X. Wang, M. Zhu, D. Bo, P. Cui, C. Shi, J. Pei, AM-GCN: Adaptive Multi-channel Graph Convolutional Networks, in: Proceedings of the 26th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2020, pp. 1243–1253.

[17]

K. Sun, Z. Lin, Z. Zhu, Multi-Stage Self-Supervised Learning for Graph Convolutional Networks on Graphs with Few Labeled Nodes, in: Proceedings of the 34th AAAI Conference on Artificial Intelligence, 2020, pp. 5892–5899.

[18]

Z. Chen, Z. Wu, S. Wang, W. Guo, Dual Low-rank Graph Autoencoder for Semantic and Topological Networks, in: Proceedings of the 37th AAAI Conference on Artificial Intelligence, 37, (4) 2023, pp. 4191–4198.

[19]

Z. Wu, L. Shu, Z. Xu, Y. Chang, C. Chen, Z. Zheng, Robust Tensor Graph Convolutional Networks via T-SVD Based Graph Augmentation, in: Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, 2022, pp. 2090–2099.

[20]

K. Sun, Z. Zhu, Z. Lin, AdaGCN: Adaboosting Graph Convolutional Networks into Deep Models, in: Proceedings of the 9th International Conference on Learning Representations, 2021, pp. 1–15.

[21]

Y. Zhang, X. Wang, C. Shi, N. Liu, G. Song, Lorentzian Graph Convolutional Networks, in: Proceedings of International World Wide Web Conference, 2021, pp. 1249–1261.

[22]

F. Wu, A.H.S. Jr., T. Zhang, C. Fifty, T. Yu, K.Q. Weinberger, Simplifying Graph Convolutional Networks, in: Proceedings of the 36th International Conference on Machine Learning, Vol. 97, 2019, pp. 6861–6871.

[23]

Wang Y., Wang Y., Yang J., Lin Z., Dissecting the diffusion process in linear graph convolutional networks, Adv. Neural Inf. Process. Syst. (2021) 5758–5769.

[24]

R. Li, S. Wang, F. Zhu, J. Huang, Adaptive Graph Convolutional Neural Networks, in: Proceedings of the 32rd AAAI Conference on Artificial Intelligence, 2018, pp. 3546–3553.

[25]

Z. Chen, C. Chen, Z. Zhang, Z. Zheng, Q. Zou, Variational Graph Embedding and Clustering with Laplacian Eigenmaps, in: Proceedings of the 28th International Joint Conference on Artificial Intelligence, 2019, pp. 2144–2150.

[26]

Wang J., Tang C., Zheng X., Liu X., Zhang W., Zhu E., Graph regularized spatial-spectral subspace clustering for hyperspectral band selection, Neural Netw. 153 (2022) 292–302.

[27]

Li Z., Tang C., Liu X., Zheng X., Zhang W., Zhu E., Consensus graph learning for multi-view clustering, IEEE Trans. Multimedia 24 (2022) 2461–2472.

[28]

Huang A., Wang Z., Zheng Y., Zhao T., Lin C., Embedding regularizer learning for multi-view semi-supervised classification, IEEE Trans. Image Process. 30 (2021) 6997–7011.

[29]

Wang S., Chen Z., Du S., Lin Z., Learning deep sparse regularizers with applications to multi-view clustering and semi-supervised classification, IEEE Trans. Pattern Anal. Mach. Intell. 44 (9) (2022) 5042–5055.

[30]

Col A.D., Petronetto F., Graph regularization multidimensional projection, Pattern Recognit. 129 (2022).

[31]

Kang Z., Peng C., Cheng Q., Liu X., Peng X., Xu Z., Tian L., Structured graph learning for clustering and semi-supervised classification, Pattern Recognit. 110 (2021).

[32]

Huang S., Zhang Y., Fu L., Wang S., Learnable multi-view matrix factorization with graph embedding and flexible loss, IEEE Trans. Multimedia 25 (2023) 3259–3272.

[33]

Huang A., Chen W., Zhao T., Chen C.W., Joint learning of latent similarity and local embedding for multi-view clustering, IEEE Trans. Image Process. 30 (2021) 6772–6784,.

Digital Library

[34]

M. Zhu, X. Wang, C. Shi, H. Ji, P. Cui, Interpreting and Unifying Graph Neural Networks with An Optimization Framework, in: Proceedings of International World Wide Web Conference, 2021, pp. 1215–1226.

[35]

F. Nie, H. Wang, H. Huang, C.H.Q. Ding, Adaptive Loss Minimization for Semi-Supervised Elastic Embedding, in: Proceedings of the 23rd International Joint Conference on Artificial Intelligence, 2013, pp. 1565–1571.

[36]

Wang Q., Gao Q., Gao X., Nie F., ℓ 2, p-Norm based PCA for image recognition, IEEE Trans. Image Process. 27 (3) (2018) 1336–1346.

[37]

K. Xu, C. Li, Y. Tian, T. Sonobe, K.-i. Kawarabayashi, S. Jegelka, Representation learning on graphs with jumping knowledge networks, in: Proceedings of the 35th International Conference on Machine Learning, 2018, pp. 5453–5462.

[38]

Defferrard M., Bresson X., Vandergheynst P., Convolutional neural networks on graphs with fast localized spectral filtering, Adv. Neural Inf. Process. Syst. 29 (2016) 3837–3845.

[39]

P. Velickovic, G. Cucurull, A. Casanova, A. Romero, P. Liò, Y. Bengio, Graph Attention Networks, in: Proceedings of the 6th International Conference on Learning Representations, 2018, pp. 1–12.

[40]

Min Y., Wenkel F., Wolf G., Scattering GCN: overcoming oversmoothness in graph convolutional networks, Adv. Neural Inf. Process. Syst. 33 (2020) 14498–14508.

[41]

H. Zhu, P. Koniusz, Simple Spectral Graph Convolution, in: Proceedings of the 9th International Conference on Learning Representations, 2021, pp. 1–15.

[42]

J. Park, S. Yoo, J. Park, H.J. Kim, Deformable Graph Convolutional Networks, in: Proceedings of the 36th AAAI Conference on Artificial Intelligence, 2022, pp. 7949–7956.

[43]

Hamilton W.L., Ying Z., Leskovec J., Inductive representation learning on large graphs, Adv. Neural Inf. Process. Syst. 30 (2017) 1024–1034.

[44]

J. Klicpera, A. Bojchevski, S. Günnemann, Predict then Propagate: Graph Neural Networks meet Personalized PageRank, in: Proceedings of the 7th International Conference on Learning Representation, 2019, pp. 1–15.

Recommendations

Graph convolutional networks of reconstructed graph structure with constrained Laplacian rank
Abstract
Convolutional neural networks (CNNs) have achieved unprecedented competitiveness in text and two-dimensional image data processing because of its good accuracy performance and high detection speed. Graph convolutional networks (GCNs), as an ...
Semi-Supervised Classification of Graph Convolutional Networks with Laplacian Rank Constraints
Abstract
Graph convolutional networks (GCNs), as an extension of classic convolutional neural networks (CNNs) in graph processing, have achieved good results in completing semi-supervised learning tasks. Traditional GCNs usually use fixed graph to complete ...
Exploiting node-feature bipartite graph in graph convolutional networks
Abstract
In recent years, Graph Convolutional Networks (GCNs), which extend convolutional neural networks to graph structure, have achieved great success on many graph learning tasks by fusing structure and feature information, such as node ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Pattern Recognition

Pattern Recognition Volume 151, Issue C

Jul 2024

745 pages

Issue’s Table of Contents

Elsevier Ltd.

Publisher

Elsevier Science Inc.

United States

Publication History

Published: 09 July 2024

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

0
Total Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 22 Sep 2024

Other Metrics

View Author Metrics

Citations

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents