Invariant Graph Neural Network for Out-of-Distribution Nodes
Authors: 
Zhengyu Chen, Yishu Gong, Liangliang Yang, Jianyu Zhang, Wei Zhang, Sihong He, Xusheng ZhangAuthors Info & Claims
Pages 192 - 196
Abstract
GNNs are effective for semi-supervised learning tasks on graphs, but they can suffer from bias due to distribution shifts between training and testing node distributions. In this paper, we propose the Invariant Graph Neural Network (IGNN) to address the issue of bias in GNNs. Specifically, IGNN learns the correlation of invariant features in different environments, where the spurious correlation changes in different environments. IGNN contains two components: the invariant graph partition component learns different graph environments and the invariant graph learning component regularizes the graph neural network to learn invariant graph representation in these environments. Extensive experiments have shown that the IGNN outperforms other methods for out-of-distribution nodes on several benchmark datasets.
References
[1]
Martin Arjovsky, Léon Bottou, Ishaan Gulrajani, and David Lopez-Paz. 2019. Invariant risk minimization. arXiv preprint arXiv:1907.02893 (2019).
[2]
Jun Bai, Bingjun Li, and Sheida Nabavi. 2022. Semi-supervised classification of disease prognosis using CR images with clinical data structured graph. In Proceedings of the 13th ACM International Conference on Bioinformatics, Computational Biology and Health Informatics. 1–9.
[3]
Zhengyu Chen, Sibo Gai, and Donglin Wang. 2019. Deep Tensor Factorization for Multi-Criteria Recommender Systems. In 2019 IEEE International Conference on Big Data (Big Data), Los Angeles, CA, USA, December 9-12, 2019. 1046–1051.
[4]
Zhengyu Chen, Jixie Ge, Heshen Zhan, Siteng Huang, and Donglin Wang. 2021. Pareto Self-Supervised Training for Few-Shot Learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 13663–13672.
[5]
Zhengyu Chen and Donglin Wang. 2021. Multi-Initialization Meta-Learning with Domain Adaptation. In ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 1390–1394.
[6]
Zhengyu Chen, Donglin Wang, and Shiqian Yin. 2021. Improving Cold-Start Recommendation via Multi-Prior Meta-Learning. In 43rd European Conference on Information Retrieval, ECIR 2021.
[7]
Zhengyu Chen, Teng Xiao, and Kun Kuang. 2022. BA-GNN: On Learning Bias-Aware Graph Neural Network. In 2022 IEEE 38th International Conference on Data Engineering (ICDE). IEEE, 3012–3024.
[8]
Zhengyu Chen, Ziqing Xu, and Donglin Wang. 2021. Deep transfer tensor decomposition with orthogonal constraint for recommender systems. In The Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI, Vol. 2021. 3.
[9]
Michaël Defferrard, Xavier Bresson, and Pierre Vandergheynst. 2016. Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering. In NIPS.
[10]
Matthias Fey and Jan Eric Lenssen. 2019. Fast graph representation learning with PyTorch Geometric. arXiv preprint arXiv:1903.02428 (2019).
[11]
Sibo Gai, Feng Zhao, Yachen Kang, Zhengyu Chen, Donglin Wang, and Ao Tang. 2019. Deep transfer collaborative filtering for recommender systems. In Pacific Rim International Conference on Artificial Intelligence. Springer, 515–528.
[12]
William L Hamilton, Rex Ying, and Jure Leskovec. 2017. Inductive representation learning on large graphs. In 31st NeurIPS. 1025–1035.
[13]
Sihong He, Songyang Han, Sanbao Su, Shuo Han, Shaofeng Zou, and Fei Miao. 2023. Robust Multi-Agent Reinforcement Learning Considering State Uncertainties. AI4ABM Workshop at the International Conference on Learning Representations (ICLR) (2023).
[14]
Sihong He, Lynn Pepin, Guang Wang, Desheng Zhang, and Fei Miao. 2020. Data-driven distributionally robust electric vehicle balancing for mobility-on-demand systems under demand and supply uncertainties. In 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2165–2172.
[15]
Sihong He, Yue Wang, Shuo Han, Shaofeng Zou, and Fei Miao. 2022. A Robust and Constrained Multi-Agent Reinforcement Learning Framework for Electric Vehicle AMoD Systems. arXiv preprint arXiv:2209.08230 (2022).
[16]
Sihong He, Zhili Zhang, Shuo Han, Lynn Pepin, Guang Wang, Desheng Zhang, John Stankovic, and Fei Miao. 2022. Data-Driven Distributionally Robust Electric Vehicle Balancing for Autonomous Mobility-on-Demand Systems under Demand and Supply Uncertainties. arXiv preprint arXiv:2211.13797 (2022).
[17]
Yinjie Jiang, Zhengyu Chen, Kun Kuang, Luotian Yuan, Xinhai Ye, Zhihua Wang, Fei Wu, and Ying Wei. 2022. The Role of Deconfounding in Meta-learning. In International Conference on Machine Learning. PMLR, 10161–10176.
[18]
Thomas N. Kipf and Max Welling. 2017. Semi-Supervised Classification with Graph Convolutional Networks. In 5th ICLR.
[19]
Johannes Klicpera, Aleksandar Bojchevski, and Stephan Günnemann. 2019. Predict then Propagate: Graph Neural Networks meet Personalized PageRank. In 7th ICLR 2019.
[20]
Bingjun Li, Tianyu Wang, and Sheida Nabavi. 2021. Cancer molecular subtype classification by graph convolutional networks on multi-omics data. In Proceedings of the 12th ACM Conference on Bioinformatics, Computational Biology, and Health Informatics. 1–9.
[21]
Julian McAuley, Christopher Targett, Qinfeng Shi, and Anton Van Den Hengel. 2015. Image-based recommendations on styles and substitutes. In Proceedings of the 38th international ACM SIGIR conference on research and development in information retrieval. 43–52.
[22]
Fei Miao, Sihong He, Lynn Pepin, Shuo Han, Abdeltawab Hendawi, Mohamed E Khalefa, John A Stankovic, and George Pappas. 2021. Data-driven distributionally robust optimization for vehicle balancing of mobility-on-demand systems. ACM Transactions on Cyber-Physical Systems 5, 2 (2021), 1–27.
[23]
Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. 2017. Automatic differentiation in pytorch. (2017).
[24]
Hongbin Pei, Bingzhe Wei, Kevin Chen-Chuan Chang, Yu Lei, and Bo Yang. 2019. Geom-GCN: Geometric Graph Convolutional Networks. In ICLR.
[25]
Prithviraj Sen, Galileo Namata, Mustafa Bilgic, Lise Getoor, Brian Galligher, and Tina Eliassi-Rad. 2008. Collective classification in network data. AI magazine 29, 3 (2008), 93–93.
[26]
Oleksandr Shchur, Maximilian Mumme, Aleksandar Bojchevski, and Stephan Günnemann. 2018. Pitfalls of graph neural network evaluation. arXiv preprint arXiv:1811.05868 (2018).
[27]
Ke Sun, Zhouchen Lin, and Zhanxing Zhu. 2019. Multi-Stage Self-Supervised Learning for Graph Convolutional Networks on Graphs with Few Labels. arXiv:1902.11038 (2019).
[28]
Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. 2018. Graph Attention Networks. In 6th ICLR.
[29]
Tianyu Wang, Bingjun Li, and Sheida Nabavi. 2021. Single-cell RNA sequencing data clustering using graph convolutional networks. In 2021 IEEE International Conference on Bioinformatics and Biomedicine (BIBM). IEEE, 2163–2170.
[30]
Teng Xiao, Zhengyu Chen, Zhimeng Guo, Zeyang Zhuang, and Suhang Wang. [n.d.]. Decoupled Self-supervised Learning for Graphs. In Advances in Neural Information Processing Systems.
[31]
Teng Xiao, Zhengyu Chen, Zhimeng Guo, Zeyang Zhuang, and Suhang Wang. 2022. Decoupled Self-supervised Learning for Non-Homophilous Graphs. arXiv preprint arXiv:2206.03601 (2022).
[32]
Teng Xiao, Zhengyu Chen, Donglin Wang, and Suhang Wang. 2021. Learning How to Propagate Messages in Graph Neural Networks. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. 1894–1903.
[33]
Teng Xiao, Zhengyu Chen, and Suhang Wang. 2022. Representation Matters When Learning From Biased Feedback in Recommendation. In Proceedings of the 31st ACM International Conference on Information & Knowledge Management. 2220–2229.
[34]
Teng Xiao, Zhengyu Chen, and Suhang Wang. 2022. Towards Bridging Algorithm and Theory for Unbiased Recommendation. arXiv preprint arXiv:2206.03851 (2022).
Index Terms
- Invariant Graph Neural Network for Out-of-Distribution Nodes
Recommendations
FLOOD: A Flexible Invariant Learning Framework for Out-of-Distribution Generalization on Graphs
KDD '23: Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data MiningGraph Neural Networks (GNNs) have achieved remarkable success in various domains but most of them are developed under the in-distribution assumption. Under out-of-distribution (OOD) settings, they suffer from the distribution shift between the training ...
Edge Based Graph Neural Network to Recognize Semigraph Representation of English Alphabets
MIKE 2013: Proceedings of the First International Conference on Mining Intelligence and Knowledge Exploration - Volume 8284Graph Neural Network based on edges is introduced in this paper and is used to recognize the English uppercase alphabets treating their corresponding graphs as semigraphs. Graph Neural Network(GNN) is a connectionist model comprising of two feedforward ...
Generalizing Graph Neural Network across Graphs and Time
WSDM '23: Proceedings of the Sixteenth ACM International Conference on Web Search and Data MiningGraph-structured data widely exist in diverse real-world scenarios, analysis of these graphs can uncover valuable insights about their respective application domains. However, most previous works focused on learning node representation from a single ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
February 2023
619 pages
ISBN:9781450398411
DOI:10.1145/3587716
Copyright © 2023 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 07 September 2023
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
ICMLC 2023
ICMLC 2023: 2023 15th International Conference on Machine Learning and Computing
February 17 - 20, 2023
Zhuhai, China
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 61Total Downloads
- Downloads (Last 12 months)51
- Downloads (Last 6 weeks)4
Reflects downloads up to 25 Nov 2024
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format