Nothing Special   »   [go: up one dir, main page]

skip to main content
10.1145/3581783.3612026acmconferencesArticle/Chapter ViewAbstractPublication PagesmmConference Proceedingsconference-collections
research-article

ALEX: Towards Effective Graph Transfer Learning with Noisy Labels

Published: 27 October 2023 Publication History

Abstract

Graph Neural Networks (GNNs) have garnered considerable interest due to their exceptional performance in a wide range of graph machine learning tasks. Nevertheless, the majority of GNN-based approaches have been examined using well-annotated benchmark datasets, leading to suboptimal performance in real-world graph learning scenarios. To bridge this gap, the present paper investigates the problem of graph transfer learning in the presence of label noise, which transfers knowledge from a noisy source graph to an unlabeled target graph. We introduce a novel technique termed Balance Alignment and Information-aware Examination (ALEX) to address this challenge. ALEX first employs singular value decomposition to generate different views with crucial structural semantics, which help provide robust node representations using graph contrastive learning. To mitigate both label shift and domain shift, we estimate a prior distribution to build subgraphs with balanced label distributions. Building on this foundation, an adversarial domain discriminator is incorporated for the implicit domain alignment of complex multi-modal distributions. Furthermore, we project node representations into a different space, optimizing the mutual information between the projected features and labels. Subsequently, the inconsistency of similarity structures is evaluated to identify noisy samples with potential overfitting. Comprehensive experiments on various benchmark datasets substantiate the outstanding superiority of the proposed ALEX in different settings.

References

[1]
Kirk Baker. 2005. Singular value decomposition tutorial. The Ohio State University, Vol. 24 (2005).
[2]
Mohamed Ishmael Belghazi, Aristide Baratin, Sai Rajeshwar, Sherjil Ozair, Yoshua Bengio, Aaron Courville, and Devon Hjelm. 2018. Mutual information neural estimation. In Proceedings of the International Conference on Machine Learning. 531--540.
[3]
Xuheng Cai, Chao Huang, Lianghao Xia, and Xubin Ren. 2023. LightGCL: Simple Yet Effective Graph Contrastive Learning for Recommendation. arXiv preprint arXiv:2302.08191 (2023).
[4]
Ji Chang, Jing Li, Yu Kang, Wenjun Lv, Ting Xu, Zerui Li, Wei Xing Zheng, Hongwei Han, and Haining Liu. 2021. Unsupervised domain adaptation using maximum mean discrepancy optimization for lithology identification. Geophysics, Vol. 86, 2 (2021), ID19-ID30.
[5]
Yongqiang Chen, Yonggang Zhang, Yatao Bian, Han Yang, MA Kaili, Binghui Xie, Tongliang Liu, Bo Han, and James Cheng. 2022. Learning causally invariant representations for out-of-distribution generalization on graphs. (2022), 22131--22148.
[6]
Enyan Dai, Charu Aggarwal, and Suhang Wang. 2021. NRGNN: Learning a label noise resistant graph neural network on sparsely and noisily labeled graphs. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. 227--236.
[7]
Quanyu Dai, Xiao-Ming Wu, Jiaren Xiao, Xiao Shen, and Dan Wang. 2022. Graph Transfer Learning via Adversarial Domain Adaptation with Graph Convolution. IEEE Transactions on Knowledge and Data Engineering (2022).
[8]
Xuefeng Du, Tian Bian, Yu Rong, Bo Han, Tongliang Liu, Tingyang Xu, Wenbing Huang, and Junzhou Huang. 2021. PI-GNN: A Novel Perspective on Semi-Supervised Node Classification against Noisy Labels. arXiv preprint arXiv:2106.07451 (2021).
[9]
Wenqi Fan, Yao Ma, Qing Li, Yuan He, Eric Zhao, Jiliang Tang, and Dawei Yin. 2019. Graph neural networks for social recommendation. In The world wide web conference. 417--426.
[10]
Wenqi Fan, Yao Ma, Qing Li, Jianping Wang, Guoyong Cai, Jiliang Tang, and Dawei Yin. 2020. A graph neural network framework for social recommendations. IEEE Transactions on Knowledge and Data Engineering, Vol. 34, 5 (2020), 2033--2047.
[11]
Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, Francc ois Laviolette, Mario Marchand, and Victor Lempitsky. 2016. Domain-adversarial training of neural networks. The journal of machine learning research, Vol. 17, 1 (2016), 2096--2030.
[12]
Yunchao Gong, Qifa Ke, Michael Isard, and Svetlana Lazebnik. 2014. A multi-view embedding space for modeling internet images, tags, and their semantics. International Journal of Computer Vision, Vol. 106 (2014), 210--233.
[13]
Nathan Halko, Per-Gunnar Martinsson, and Joel A Tropp. 2011. Finding structure with randomness: Probabilistic algorithms for constructing approximate matrix decompositions. SIAM Rev., Vol. 53, 2 (2011), 217--288.
[14]
William L Hamilton, Rex Ying, and Jure Leskovec. 2017. Inductive representation learning on large graphs. In Proceedings of the 31st International Conference on Neural Information Processing Systems. 1025--1035.
[15]
Bo Han, Quanming Yao, Xingrui Yu, Gang Niu, Miao Xu, Weihua Hu, Ivor Tsang, and Masashi Sugiyama. 2018. Co-teaching: Robust training of deep neural networks with extremely noisy labels. In Proceedings of the Conference on Neural Information Processing Systems.
[16]
Lei Ji, Rongcheng Tu, Kevin Lin, Lijuan Wang, and Nan Duan. 2022. Multimodal graph neural network for video procedural captioning. Neurocomputing, Vol. 488 (2022), 88--96.
[17]
Xiang Jiang, Qicheng Lao, Stan Matwin, and Mohammad Havaei. 2020. Implicit class-conditioned domain alignment for unsupervised domain adaptation. In Proceedings of the International Conference on Machine Learning. 4816--4827.
[18]
Wei Ju, Zheng Fang, Yiyang Gu, Zequn Liu, Qingqing Long, Ziyue Qiao, Yifang Qin, Jianhao Shen, Fang Sun, Zhiping Xiao, et al. 2023 a. A Comprehensive Survey on Deep Graph Representation Learning. arXiv preprint arXiv:2304.05055 (2023).
[19]
Wei Ju, Xiao Luo, Meng Qu, Yifan Wang, Chong Chen, Minghua Deng, Xian-Sheng Hua, and Ming Zhang. 2023 b. TGNN: A joint semi-supervised framework for graph-level classification. arXiv preprint arXiv:2304.11688 (2023).
[20]
Wei Ju, Yifang Qin, Ziyue Qiao, Xiao Luo, Yifan Wang, Yanjie Fu, and Ming Zhang. 2022. Kernel-based substructure exploration for next POI recommendation. In 2022 IEEE International Conference on Data Mining (ICDM). IEEE, 221--230.
[21]
Thomas N Kipf and Max Welling. 2017. Semi-supervised classification with graph convolutional networks. In ICLR.
[22]
Jisoo Lee and Sae-Young Chung. 2020. Robust training with ensemble consensus. In Proceedings of the International Conference on Learning Representations.
[23]
Jinfeng Li, Weifeng Liu, Yicong Zhou, Jun Yu, Dapeng Tao, and Changsheng Xu. 2022. Domain-invariant graph for adaptive semi-supervised domain adaptation. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM), Vol. 18, 3 (2022), 1--18.
[24]
Mengxue Li, Yi-Ming Zhai, You-Wei Luo, Peng-Fei Ge, and Chuan-Xian Ren. 2020. Enhanced transport distance for unsupervised domain adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 13936--13944.
[25]
Shuang Li, Chi Harold Liu, Binhui Xie, Limin Su, Zhengming Ding, and Gao Huang. 2019. Joint adversarial domain adaptation. In Proceedings of the ACM International Conference on Multimedia. 729--737.
[26]
Xiaofeng Liu, Zhenhua Guo, Site Li, Fangxu Xing, Jane You, C-C Jay Kuo, Georges El Fakhri, and Jonghye Woo. 2021. Adversarial unsupervised domain adaptation with conditional and label shift: Infer, align and iterate. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 10367--10376.
[27]
Mingsheng Long, Zhangjie Cao, Jianmin Wang, and Michael I Jordan. 2018. Conditional adversarial domain adaptation. Advances in neural information processing systems, Vol. 31 (2018).
[28]
Xiao Luo, Wei Ju, Meng Qu, Yiyang Gu, Chong Chen, Minghua Deng, Xian-Sheng Hua, and Ming Zhang. 2022. Clear: Cluster-enhanced contrast for self-supervised graph representation learning. IEEE Transactions on Neural Networks and Learning Systems (2022).
[29]
Xiao Luo, Daqing Wu, Zeyu Ma, Chong Chen, Minghua Deng, Jinwen Ma, Zhongming Jin, Jianqiang Huang, and Xian-Sheng Hua. 2020b. Cimon: Towards high-quality hash codes. arXiv preprint arXiv:2010.07804 (2020).
[30]
Xiao Luo, Jingyang Yuan, Zijie Huang, Huiyu Jiang, Yifang Qin, Wei Ju, Ming Zhang, and Yizhou Sun. 2023. HOPE: High-order Graph ODE For Modeling Interacting Dynamics. (2023).
[31]
Yadan Luo, Zi Huang, Zijian Wang, Zheng Zhang, and Mahsa Baktashmotlagh. 2020a. Adversarial bipartite graph learning for video domain adaptation. In Proceedings of the ACM International Conference on Multimedia. 19--27.
[32]
Hoang NT, Choong Jun Jin, and Tsuyoshi Murata. 2019. Learning graph neural networks with noisy labels. arXiv preprint arXiv:1905.01591 (2019).
[33]
Youngmin Oh, Beomjun Kim, and Bumsub Ham. 2021. Background-Aware Pooling and Noise-Aware Loss for Weakly-Supervised Semantic Segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.
[34]
Giorgio Patrini, Alessandro Rozza, Aditya Krishna Menon, Richard Nock, and Lizhen Qu. 2017. Making deep neural networks robust to label noise: A loss correction approach. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.
[35]
Shichao Pei, Lu Yu, Guoxian Yu, and Xiangliang Zhang. 2020. Rea: Robust cross-lingual entity alignment between knowledge graphs. In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2175--2184.
[36]
Bryan Perozzi, Rami Al-Rfou, and Steven Skiena. 2014. Deepwalk: Online learning of social representations. In Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining. 701--710.
[37]
Siyi Qian, Haochao Ying, Renjun Hu, Jingbo Zhou, Jintai Chen, Danny Z Chen, and Jian Wu. 2023. Robust training of graph neural networks via noise governance. In Proceedings of the Sixteenth ACM International Conference on Web Search and Data Mining. 607--615.
[38]
Scott Reed, Honglak Lee, Dragomir Anguelov, Christian Szegedy, Dumitru Erhan, and Andrew Rabinovich. 2015. Training deep neural networks on noisy labels with bootstrapping. In Proceedings of the International Conference on Learning Representations.
[39]
Amirreza Salamat, Xiao Luo, and Ali Jafari. 2021. HeteroGraphRec: A heterogeneous graph-based neural networks for social recommendations. Knowledge-Based Systems, Vol. 217 (2021), 106817.
[40]
Xiao Shen, Quanyu Dai, Sitong Mao, Fu-lai Chung, and Kup-Sze Choi. 2020. Network together: Node classification via cross-network deep network embedding. IEEE Transactions on Neural Networks and Learning Systems, Vol. 32, 5 (2020), 1935--1948.
[41]
Cheng Tan, Jun Xia, Lirong Wu, and Stan Z Li. 2021. Co-learning: Learning from noisy labels with self-supervision. In Proceedings of the ACM International Conference on Multimedia. 1405--1413.
[42]
Jian Tang, Meng Qu, Mingzhe Wang, Ming Zhang, Jun Yan, and Qiaozhu Mei. 2015. Line: Large-scale information network embedding. In Proceedings of the 24th international conference on world wide web. 1067--1077.
[43]
Jie Tang, Jing Zhang, Limin Yao, Juanzi Li, Li Zhang, and Zhong Su. 2008. Arnetminer: extraction and mining of academic social networks. In Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining. 990--998.
[44]
Eric Tzeng, Judy Hoffman, Kate Saenko, and Trevor Darrell. 2017. Adversarial discriminative domain adaptation. In Proceedings of the IEEE conference on computer vision and pattern recognition. 7167--7176.
[45]
Petar Velivc ković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. 2018. Graph Attention Networks. In International Conference on Learning Representations.
[46]
Chenyu Wang, Zongyu Lin, Xiaochen Yang, Jiao Sun, Mingxuan Yue, and Cyrus Shahabi. 2022. Hagen: Homophily-aware graph convolutional recurrent network for crime forecasting. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 36. 4193--4200.
[47]
Qifan Wang, Yinwei Wei, Jianhua Yin, Jianlong Wu, Xuemeng Song, and Liqiang Nie. 2021c. Dualgnn: Dual graph neural network for multimedia recommendation. IEEE Transactions on Multimedia (2021).
[48]
Tiantian Wang, Sifei Liu, Yapeng Tian, Kai Li, and Ming-Hsuan Yang. 2021b. Video matting via consistency-regularized graph neural networks. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 4902--4911.
[49]
Wenguan Wang, Xiankai Lu, Jianbing Shen, David J Crandall, and Ling Shao. 2019. Zero-shot video object segmentation via attentive graph neural networks. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 9236--9245.
[50]
Xiao Wang, Hongrui Liu, Chuan Shi, and Cheng Yang. 2021a. Be confident! towards trustworthy graph neural networks via confidence calibration. In Proceedings of the Conference on Neural Information Processing Systems. 23768--23779.
[51]
Felix Wu, Amauri Souza, Tianyi Zhang, Christopher Fifty, Tao Yu, and Kilian Weinberger. 2019. Simplifying graph convolutional networks. In ICML.
[52]
Man Wu, Shirui Pan, Chuan Zhou, Xiaojun Chang, and Xingquan Zhu. 2020b. Unsupervised domain adaptive graph convolutional networks. In Proceedings of The Web Conference 2020. 1457--1467.
[53]
Man Wu, Shirui Pan, and Xingquan Zhu. 2022a. Attraction and Repulsion: Unsupervised Domain Adaptive Graph Contrastive Learning Network. IEEE Transactions on Emerging Topics in Computational Intelligence (2022).
[54]
Ying-Xin Wu, Xiang Wang, An Zhang, Xiangnan He, and Tat-Seng Chua. 2022b. Discovering invariant rationales for graph neural networks. In Proceedings of the International Conference on Machine Learning.
[55]
Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long, Chengqi Zhang, and S Yu Philip. 2020a. A comprehensive survey on graph neural networks. IEEE transactions on neural networks and learning systems (2020).
[56]
Louis-Pascal Xhonneux, Meng Qu, and Jian Tang. 2020. Continuous graph neural networks. In International Conference on Machine Learning. PMLR, 10432--10441.
[57]
Zizheng Yan, Yushuang Wu, Guanbin Li, Yipeng Qin, Xiaoguang Han, and Shuguang Cui. 2022. Multi-level Consistency Learning for Semi-supervised Domain Adaptation. In Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence, IJCAI-22, Lud De Raedt (Ed.). International Joint Conferences on Artificial Intelligence Organization, 1530--1536. https://doi.org/10.24963/ijcai.2022/213 Main Track.
[58]
Xingrui Yu, Bo Han, Jiangchao Yao, Gang Niu, Ivor Tsang, and Masashi Sugiyama. 2019. How does disagreement help generalization against label corruption?. In International Conference on Machine Learning. PMLR, 7164--7173.
[59]
Hao Yuan, Jiliang Tang, Xia Hu, and Shuiwang Ji. 2020. Xgnn: Towards model-level explanations of graph neural networks. In Proceedings of the International ACM SIGKDD Conference on Knowledge Discovery & Data Mining. 430--438.
[60]
Jingyang Yuan, Xiao Luo, Yifang Qin, Yusheng Zhao, Wei Ju, and Ming Zhang. 2023. Learning on Graphs under Label Noise. In IEEE International Conference on Acoustics, Speech and Signal Processing. 1--5.
[61]
Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. 2021a. Understanding deep learning (still) requires rethinking generalization. Commun. ACM (2021).
[62]
HaiYang Zhang, XiMing Xing, and Liang Liu. 2021c. Dualgraph: A graph-based method for reasoning about label noise. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 9654--9663.
[63]
Weichen Zhang, Wanli Ouyang, Wen Li, and Dong Xu. 2018. Collaborative and adversarial network for unsupervised domain adaptation. In Proceedings of the IEEE conference on computer vision and pattern recognition. 3801--3809.
[64]
Wenyu Zhang, Li Shen, Wanyue Zhang, and Chuan-Sheng Foo. 2022b. Few-Shot Adaptation of Pre-Trained Networks for Domain Shift. In Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence, IJCAI-22, Lud De Raedt (Ed.). International Joint Conferences on Artificial Intelligence Organization, 1665--1671. https://doi.org/10.24963/ijcai.2022/232 Main Track.
[65]
Xiaowen Zhang, Yuntao Du, Rongbiao Xie, and Chongjun Wang. 2021b. Adversarial Separation Network for Cross-Network Node Classification. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management. 2618--2626.
[66]
Yanfu Zhang, Shangqian Gao, Jian Pei, and Heng Huang. 2022a. Improving Social Network Embedding via New Second-Order Continuous Graph Neural Networks. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 2515--2523.
[67]
Qi Zhu, Carl Yang, Yidan Xu, Haonan Wang, Chao Zhang, and Jiawei Han. 2021. Transfer learning of graph neural networks with ego-graph information maximization. In Proceedings of the Conference on Neural Information Processing Systems. 1766--1779.

Cited By

View all
  • (2025)Rethinking the impact of noisy labels in graph classification: A utility and privacy perspectiveNeural Networks10.1016/j.neunet.2024.106919182(106919)Online publication date: Feb-2025
  • (2024)A Comprehensive Survey on Deep Graph Representation LearningNeural Networks10.1016/j.neunet.2024.106207173:COnline publication date: 2-Jul-2024
  • (2024)Partial label learning via weighted centroid clustering disambiguationNeurocomputing10.1016/j.neucom.2024.128312604(128312)Online publication date: Nov-2024
  • Show More Cited By

Recommendations

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences
MM '23: Proceedings of the 31st ACM International Conference on Multimedia
October 2023
9913 pages
ISBN:9798400701085
DOI:10.1145/3581783
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 October 2023

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. domain adaptation
  2. graph neural networks
  3. graph transfer learning
  4. label noise

Qualifiers

  • Research-article

Funding Sources

Conference

MM '23
Sponsor:
MM '23: The 31st ACM International Conference on Multimedia
October 29 - November 3, 2023
Ottawa ON, Canada

Acceptance Rates

Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)285
  • Downloads (Last 6 weeks)22
Reflects downloads up to 20 Nov 2024

Other Metrics

Citations

Cited By

View all
  • (2025)Rethinking the impact of noisy labels in graph classification: A utility and privacy perspectiveNeural Networks10.1016/j.neunet.2024.106919182(106919)Online publication date: Feb-2025
  • (2024)A Comprehensive Survey on Deep Graph Representation LearningNeural Networks10.1016/j.neunet.2024.106207173:COnline publication date: 2-Jul-2024
  • (2024)Partial label learning via weighted centroid clustering disambiguationNeurocomputing10.1016/j.neucom.2024.128312604(128312)Online publication date: Nov-2024
  • (2024)Supervised contrastive learning for graph representation enhancementNeurocomputing10.1016/j.neucom.2024.127710588:COnline publication date: 17-Jul-2024
  • (2024)Noise-resistant graph neural networks with manifold consistency and label consistencyExpert Systems with Applications: An International Journal10.1016/j.eswa.2023.123120245:COnline publication date: 2-Jul-2024

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media