research-article

Robust Tensor Graph Convolutional Networks via T-SVD based Graph Augmentation

Authors:

Zibin ZhengAuthors Info & Claims

KDD '22: Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

Pages 2090 - 2099

https://doi.org/10.1145/3534678.3539436

Published: 14 August 2022 Publication History

Abstract

Graph Neural Networks (GNNs) have exhibited their powerful ability of tackling nontrivial problems on graphs. However, as an extension of deep learning models to graphs, GNNs are vulnerable to noise or adversarial attacks due to the underlying perturbations propagating in message passing scheme, which can affect the ultimate performances dramatically. Thus, it's vital to study a robust GNN framework to defend against various perturbations. In this paper, we propose a Robust Tensor Graph Convolutional Network (RT-GCN) model to improve the robustness. On the one hand, we utilize multi-view augmentation to reduce the augmentation variance and organize them as a third-order tensor, followed by the truncated T-SVD to capture the low-rankness of the multi-view augmented graph, which improves the robustness from the perspective of graph preprocessing. On the other hand, to effectively capture the inter-view and intra-view information on the multi-view augmented graph, we propose tensor GCN (TGCN) framework and analyze the mathematical relationship between TGCN and vanilla GCN, which improves the robustness from the perspective of model architecture. Extensive experimental results have verified the effectiveness of RT-GCN on various datasets, demonstrating the superiority to the state-of-the-art models on diverse adversarial attacks for graphs.

Supplemental Material

MP4 File

Presentation video for KDD'2022 paper titled 'Robust Tensor Graph Convolutional Networks via T-SVD based Graph Augmentation'.

Download
41.42 MB

References

[1]

Aleksandar Bojchevski and Stephan Günnemann. 2017. Deep gaussian embedding of graphs: Unsupervised inductive learning via ranking. arXiv preprint arXiv:1707.03815 (2017).

[2]

Emmanuel J Candès, Xiaodong Li, Yi Ma, and JohnWright. 2011. Robust principal component analysis? Journal of the ACM (JACM) 58, 3 (2011), 1--37.

Digital Library

[3]

Chuan Chen, Zhe-BinWu, Zi-Tai Chen, Zi-Bin Zheng, and Xiong-Jun Zhang. 2021. Auto-weighted robust low-rank tensor completion via tensor-train. Information Sciences 567 (2021), 100--115.

[4]

Liang Chen, Jintang Li, Jiaying Peng, Tao Xie, Zengxu Cao, Kun Xu, Xiangnan He, and Zibin Zheng. 2020. A survey of adversarial learning on graphs. arXiv preprint arXiv:2003.05730 (2020).

[5]

Hanjun Dai, Hui Li, Tian Tian, Xin Huang, Lin Wang, Jun Zhu, and Le Song. 2018. Adversarial attack on graph structured data. In International conference on machine learning. PMLR, 1115--1124.

[6]

Travis Ebesu and Yi Fang. 2017. Neural citation network for context-aware citation recommendation. In Proceedings of the 40th international ACM SIGIR conference on research and development in information retrieval. 1093--1096.

Digital Library

[7]

Negin Entezari, Saba A Al-Sayouri, Amirali Darvishzadeh, and Evangelos E. Papalexakis. 2020. All you need is low (rank) defending against adversarial attacks on graphs. In Proceedings of the 13th International Conference on Web Search and Data Mining. 169--177.

[8]

C Lee Giles, Kurt D Bollacker, and Steve Lawrence. 1998. CiteSeer: An automatic citation indexing system. In Proceedings of the third ACM conference on Digital libraries. 89--98.

Digital Library

[9]

William L Hamilton, Rex Ying, and Jure Leskovec. 2017. Inductive representation learning on large graphs. In Proceedings of the 31st International Conference on Neural Information Processing Systems. 1025--1035.

[10]

Zhichao Huang, Xutao Li, Yunming Ye, and Michael K Ng. 2020. MR-GCN: Multi-Relational Graph Convolutional Networks based on Generalized Tensor Product. In IJCAI. 1258--1264.

[11]

Prateek Jain, Praneeth Netrapalli, and Sujay Sanghavi. 2013. Low-rank matrix completion using alternating minimization. In Proceedings of the forty-fifth annual ACM symposium on Theory of computing. 665--674.

Digital Library

[12]

Wei Jin, Yao Ma, Xiaorui Liu, Xianfeng Tang, Suhang Wang, and Jiliang Tang. 2020. Graph structure learning for robust graph neural networks. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 66--74.

Digital Library

[13]

Misha E Kilmer, Karen Braman, Ning Hao, and Randy C Hoover. 2013. Third-order tensors as operators on matrices: A theoretical and computational framework with applications in imaging. SIAM J. Matrix Anal. Appl. 34, 1 (2013), 148--172.

Digital Library

[14]

Misha E Kilmer and Carla D Martin. 2011. Factorization strategies for third-order tensors. Linear Algebra Appl. 435, 3 (2011), 641--658.

[15]

Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).

[16]

Thomas N Kipf and MaxWelling. 2016. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 (2016).

[17]

Xiangyu Li, Weizheng Chen, Yang Chen, Xuegong Zhang, Jin Gu, and Michael Q Zhang. 2017. Network embedding-based representation learning for single cell RNA-seq data. Nucleic acids research (2017).

[18]

Yaxin Li, Wei Jin, Han Xu, and Jiliang Tang. 2020. Deeprobust: A pytorch library for adversarial attacks and defenses. arXiv preprint arXiv:2005.06149 (2020).

[19]

Lizi Liao, Xiangnan He, Hanwang Zhang, and Tat-Seng Chua. 2018. Attributed social network embedding. IEEE Transactions on Knowledge and Data Engineering 30, 12 (2018), 2257--2270.

Digital Library

[20]

Ji Liu, Przemyslaw Musialski, Peter Wonka, and Jieping Ye. 2012. Tensor completion for estimating missing values in visual data. IEEE transactions on pattern analysis and machine intelligence 35, 1 (2012), 208--220.

[21]

Ziyang Liu, Zhaomeng Cheng, Yunjiang Jiang, Yue Shang, Wei Xiong, Sulong Xu, Bo Long, and Di Jin. 2021. Heterogeneous Network Embedding for Deep Semantic Relevance Match in E-commerce Search. arXiv preprint arXiv:2101.04850 (2021).

[22]

Canyi Lu, Jiashi Feng, Yudong Chen, Wei Liu, Zhouchen Lin, and Shuicheng Yan. 2016. Tensor robust principal component analysis: Exact recovery of corrupted low-rank tensors via convex optimization. In Proceedings of the IEEE conference on computer vision and pattern recognition. 5249--5257.

[23]

Osman Asif Malik, Shashanka Ubaru, Lior Horesh, Misha E Kilmer, and Haim Avron. 2021. Dynamic Graph Convolutional Networks Using the Tensor MProduct. In Proceedings of the 2021 SIAM International Conference on Data Mining (SDM). SIAM, 729--737.

[24]

Andrew Kachites McCallum, Kamal Nigam, Jason Rennie, and Kristie Seymore. 2000. Automating the construction of internet portals with machine learning. Information Retrieval 3, 2 (2000), 127--163.

Digital Library

[25]

Hogun Park and Jennifer Neville. 2019. Exploiting Interaction Links for Node Classification with Deep Graph Neural Networks. In IJCAI. 3223--3230.

[26]

Chanathip Pornprasit, Xin Liu, Natthawut Kertkeidkachorn, Kyoung-Sook Kim, Thanapon Noraset, and Suppawong Tuarob. 2020. Convcn: A cnn-based citation network embedding algorithm towards citation recommendation. In Proceedings of the ACM/IEEE Joint Conference on Digital Libraries in 2020. 433--436.

Digital Library

[27]

Yu Rong, Wenbing Huang, Tingyang Xu, and Junzhou Huang. 2019. Dropedge: Towards deep graph convolutional networks on node classification. arXiv preprint arXiv:1907.10903 (2019).

[28]

Ryoma Sato. 2020. A survey on the expressive power of graph neural networks. arXiv preprint arXiv:2003.04078 (2020).

[29]

Guangjing Song, Michael K Ng, and Xiongjun Zhang. 2020. Robust tensor completion using transformed tensor singular value decomposition. Numerical Linear Algebra with Applications 27, 3 (2020), e2299.

[30]

Chang Su, Jie Tong, Yongjun Zhu, Peng Cui, and Fei Wang. 2020. Network embedding in biomedical data science. Briefings in bioinformatics 21, 1 (2020), 182--197.

[31]

Laurens Van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE. Journal of machine learning research 9, 11 (2008).

[32]

Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. 2017. Graph attention networks. arXiv preprint arXiv:1710.10903 (2017).

[33]

JizheWang, Pipei Huang, Huan Zhao, Zhibo Zhang, Binqiang Zhao, and Dik Lun Lee. 2018. Billion-scale commodity embedding for e-commerce recommendation in alibaba. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 839--848.

[34]

Suhang Wang, Jiliang Tang, Charu Aggarwal, Yi Chang, and Huan Liu. 2017. Signed network embedding in social media. In Proceedings of the 2017 SIAM international conference on data mining. SIAM, 327--335.

[35]

Felix Wu, Amauri Souza, Tianyi Zhang, Christopher Fifty, Tao Yu, and Kilian Weinberger. 2019. Simplifying graph convolutional networks. In International conference on machine learning. PMLR, 6861--6871.

[36]

HuijunWu, ChenWang, Yuriy Tyshetskiy, Andrew Docherty, Kai Lu, and Liming Zhu. 2019. Adversarial examples on graph data: Deep insights into attack and defense. arXiv preprint arXiv:1903.01610 (2019).

[37]

Xiang Zhang and Marinka Zitnik. 2020. Gnnguard: Defending graph neural networks against adversarial attacks. arXiv preprint arXiv:2006.08149 (2020).

[38]

Zemin Zhang and Shuchin Aeron. 2016. Exact tensor completion using t-SVD. IEEE Transactions on Signal Processing 65, 6 (2016), 1511--1526.

Digital Library

[39]

Xile Zhao, Jinghua Yang, Tianhui Ma, Taixiang Jiang, Michael K Ng, and Tingzhu Huang. 2021. Tensor Completion via Complementary Global, Local, and Nonlocal Priors. IEEE Transactions on Image Processing (2021).

[40]

Zhun Zhong, Liang Zheng, Guoliang Kang, Shaozi Li, and Yi Yang. 2020. Random erasing data augmentation. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34. 13001--13008.

[41]

Jun Zhou, Chaochao Chen, Longfei Zheng, Xiaolin Zheng, Bingzhe Wu, Ziqi Liu, and Li Wang. 2020. Privacy-preserving graph neural network for node classification. arXiv e-prints (2020), arXiv--2005.

[42]

Ke Zhou, Hongyuan Zha, and Le Song. 2013. Learning social infectivity in sparse low-rank networks using multi-dimensional hawkes processes. In Artificial Intelligence and Statistics. PMLR, 641--649.

[43]

Pan Zhou, Canyi Lu, Zhouchen Lin, and Chao Zhang. 2017. Tensor factorization for low-rank tensor completion. IEEE Transactions on Image Processing 27, 3 (2017), 1152--1163.

[44]

Dingyuan Zhu, Ziwei Zhang, Peng Cui, and Wenwu Zhu. 2019. Robust graph convolutional networks against adversarial attacks. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 1399--1407.

Digital Library

[45]

Daniel Zügner, Amir Akbarnejad, and Stephan Günnemann. 2018. Adversarial attacks on neural networks for graph data. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2847--2856.

Digital Library

[46]

Daniel Zügner and Stephan Günnemann. 2019. Adversarial attacks on graph neural networks via meta learning. arXiv preprint arXiv:1902.08412 (2019).

Cited By

Liu XTang JLi MHan JXiao GJiang J(2024)SeSICL: Semantic and Structural Integrated Contrastive Learning for Knowledge Graph Error DetectionIEEE Access10.1109/ACCESS.2024.338454312(56088-56096)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3384543
Paul SSaha AHasan MNoori SMoustafa A(2024)A Systematic Review of Graph Neural Network in Healthcare-Based Applications: Recent Advances, Trends, and Future DirectionsIEEE Access10.1109/ACCESS.2024.335480912(15145-15170)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3354809
Wu ZChen ZDu SHuang SWang S(2024)Graph Convolutional Network with elastic topologyPattern Recognition10.1016/j.patcog.2024.110364151:COnline publication date: 9-Jul-2024
https://dl.acm.org/doi/10.1016/j.patcog.2024.110364
Show More Cited By

Index Terms

Robust Tensor Graph Convolutional Networks via T-SVD based Graph Augmentation
1. Computing methodologies
  1. Machine learning
    1. Machine learning approaches
      1. Neural networks

Recommendations

Graph Structure Learning for Robust Graph Neural Networks
KDD '20: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining

Graph Neural Networks (GNNs) are powerful tools in representation learning for graphs. However, recent studies show that GNNs are vulnerable to carefully-crafted perturbations, called adversarial attacks. Adversarial attacks can easily fool GNNs in ...
Robust Mid-Pass Filtering Graph Convolutional Networks
WWW '23: Proceedings of the ACM Web Conference 2023

Graph convolutional networks (GCNs) are currently the most promising paradigm for dealing with graph-structure data, while recent studies have also shown that GCNs are vulnerable to adversarial attacks. Thus developing GCN models that are robust to such ...
SGCN: A Graph Sparsifier Based on Graph Convolutional Networks
Advances in Knowledge Discovery and Data Mining
Abstract
Graphs are ubiquitous across the globe and within science and engineering. With graphs growing in size, node classification on large graphs can be space and time consuming, even with powerful classifiers such as Graph Convolutional Networks (GCNs)...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

KDD '22: Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

August 2022

5033 pages

ISBN:9781450393850

DOI:10.1145/3534678

General Chairs:
Aidong Zhang
University of Virginia
,
Huzefa Rangwala
Amazon/George Mason University

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 14 August 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

the Key-Area Research and Development Program of Guangdong Province
the Guangdong Basic and Applied Basic Research Foundation
the National Natural Science Foundation of China
the Tencent Wechat Rhino-bird project

Conference

KDD '22

Sponsor:

KDD '22: The 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

August 14 - 18, 2022

Washington DC, USA

Acceptance Rates

Overall Acceptance Rate 1,133 of 8,635 submissions, 13%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
558
Total Downloads

Downloads (Last 12 months)146
Downloads (Last 6 weeks)14

Reflects downloads up to 22 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Liu XTang JLi MHan JXiao GJiang J(2024)SeSICL: Semantic and Structural Integrated Contrastive Learning for Knowledge Graph Error DetectionIEEE Access10.1109/ACCESS.2024.338454312(56088-56096)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3384543
Paul SSaha AHasan MNoori SMoustafa A(2024)A Systematic Review of Graph Neural Network in Healthcare-Based Applications: Recent Advances, Trends, and Future DirectionsIEEE Access10.1109/ACCESS.2024.335480912(15145-15170)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3354809
Wu ZChen ZDu SHuang SWang S(2024)Graph Convolutional Network with elastic topologyPattern Recognition10.1016/j.patcog.2024.110364151:COnline publication date: 9-Jul-2024
https://dl.acm.org/doi/10.1016/j.patcog.2024.110364
Chen ZWu ZZhong LPlant CWang SGuo W(2024)Attributed Multi-Order Graph Convolutional Network for Heterogeneous GraphsNeural Networks10.1016/j.neunet.2024.106225174:COnline publication date: 9-Jul-2024
https://dl.acm.org/doi/10.1016/j.neunet.2024.106225
Wu ZZhang ZFan JOh ANaumann TGloberson ASaenko KHardt MLevine S(2023)Graph convolutional kernel machine versus graph convolutional networksProceedings of the 37th International Conference on Neural Information Processing Systems10.5555/3666122.3666985(19650-19672)Online publication date: 10-Dec-2023
https://dl.acm.org/doi/10.5555/3666122.3666985
Wang ALi CBai MJin ZZhou GZhao QOh ANaumann TGloberson ASaenko KHardt MLevine S(2023)Transformed low-rank parameterization can help robust generalization for tensor neural networksProceedings of the 37th International Conference on Neural Information Processing Systems10.5555/3666122.3666257(3032-3082)Online publication date: 10-Dec-2023
https://dl.acm.org/doi/10.5555/3666122.3666257
Jin KChen LYu JLu Z(2023)Robust Subgraph Augmentation for Graph Convolutional Networks with Few Labeled Nodes2023 International Conference on Advanced Robotics and Mechatronics (ICARM)10.1109/ICARM58088.2023.10218877(99-104)Online publication date: 8-Jul-2023
https://doi.org/10.1109/ICARM58088.2023.10218877

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents