Nothing Special   »   [go: up one dir, main page]
Skip to main content
Springer Nature Link
Log in

Self-supervised robust Graph Neural Networks against noisy graphs and noisy labels

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

In the paper, we first explore a novel problem of training the robust Graph Neural Networks (GNNs) against noisy graphs and noisy labels. To the problem, we propose a general Self-supervised Robust Graph Neural Network framework that consists of three modules: graph structure learning, sample selection, and self-supervised learning. Specifically, we first employ a graph structure learning approach to obtain an optimal graph structure. Next, using this structure, we use a clustering algorithm to generate pseudo-labels that represent the clusters. We then design a sample selection strategy based on these pseudo-labels to select nodes with clean labels. Additionally, we introduce a self-supervised learning technique where low-level layer parameters are shared with GNNs to predict pseudo-labels. We jointly train the graph structure learning module, the GNNs model, and the self-supervised model. Finally, we conduct extensive experiments on four real-world datasets, demonstrating the superiority of our methods compared with state-of-the-art methods for semi-supervised node classification under noisy graphs and noisy labels.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Algorithm 1
Algorithm 2
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Kipf TN, Welling M (2017) Semi-supervised classification with graph convolutional networks. In: International Conference on Learning Representations, ICLR

  2. Veličković P, Cucurull G, Casanova A, Romero A, Liò P, Bengio Y (2018) Graph attention networks. In: International Conference on Learning Representations, ICLR

  3. Hamilton W, Ying Z, Leskovec J (2017) Inductive representation learning on large graphs. In: Advances in Neural Information Processing Systems, pp 1024–1034

  4. Chien E, Peng J, Li P, Milenkovic O (2021) Adaptive universal generalized pagerank graph neural network. In: International Conference on Learning Representations, ICLR

  5. Li K, Ye W (2022) Semi-supervised node classification via graph learning convolutional neural network. Appl Intell 52(11):12724–12736

    Article  Google Scholar 

  6. Chen J, Gong Z, Wang W, Wang C, Xu Z, Lv J, Li X, Wu K, Liu W (2021) Adversarial caching training: Unsupervised inductive network representation learning on large-scale graphs. IEEE Transactions on Neural Networks and Learning Systems 33(12):7079–7090

    Article  Google Scholar 

  7. Bianchi FM, Grattarola D, Livi L, Alippi C (2022) Hierarchical representation learning in graph neural networks with node decimation pooling. IEEE Transactions on Neural Networks and Learning Systems 33(5):2195–2207

    Article  MathSciNet  Google Scholar 

  8. Tang H, Ma G, He L, Huang H, Zhan L (2021) Commpool: An interpretable graph pooling framework for hierarchical graph representation learning. Neural Netw 143:669–677

    Article  Google Scholar 

  9. Ju W, Luo X, Ma Z, Yang J, Deng M, Zhang M (2022) Ghnn: Graph harmonic neural networks for semi-supervised graph-level classification. Neural Netw 151:70–79

    Article  Google Scholar 

  10. He X, Deng K, Wang X, Li Y, Zhang Y, Wang M (2020) Lightgcn: Simplifying and powering graph convolution network for recommendation. In: Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, pp 639–648

  11. Wei T, Chow TW, Ma J, Zhao M (2023) Expgcn: Review-aware graph convolution network for explainable recommendation. Neural Netw 157:202–215

    Article  Google Scholar 

  12. Liu Y, Ma H, Jiang Y, Li Z (2022) Modelling risk and return awareness for p2p lending recommendation with graph convolutional networks. Appl Intell 52(5):4999–5014

    Article  Google Scholar 

  13. Veličković P, Fedus W, Hamilton WL, Liò P, Bengio Y, Hjelm RD (2018) Deep graph infomax. In: International Conference on Learning Representations, ICLR

  14. Klicpera J, Bojchevski A, Günnemann S (2018) Predict then propagate: Graph neural networks meet personalized pagerank. In: International Conference on Learning Representations, ICLR

  15. Liu M, Gao H, Ji S (2020) Towards deeper graph neural networks. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp 338–348

  16. Chen M, Wei Z, Huang Z, Ding B, Li Y (2020) Simple and deep graph convolutional networks. In: International Conference on Machine Learning, pp 1725–1735. PMLR

  17. Lin X, Zhou C, Wu J, Yang H, Wang H, Cao Y, Wang B (2023) Exploratory adversarial attacks on graph neural networks for semi-supervised node classification. Pattern Recogn 133:109042

    Article  Google Scholar 

  18. Dong H, Chen J, Feng F, He X, Bi S, Ding Z, Cui P (2021) On the equivalence of decoupled graph convolution network and label propagation. Proceedings of the Web Conference 2021, pp 3651–3662

  19. Zhang X, Zitnik M (2020) Gnnguard: Defending graph neural networks against adversarial attacks. Advances in Neural Information Processing Systems

  20. Jin W, Ma Y, Liu X, Tang X, Wang S, Tang J (2020) Graph structure learning for robust graph neural networks. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp 66–74

  21. Wu H, Wang C, Tyshetskiy Y, Docherty A, Lu K, Zhu L (2019) Adversarial examples on graph data: Deep insights into attack and defense. In: Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence

  22. Entezari N, Al-Sayouri SA, Darvishzadeh A, Papalexakis EE (2020) All you need is low (rank) defending against adversarial attacks on graphs. In: Proceedings of the 13th International Conference on Web Search and Data Mining, pp 169–177

  23. Dai E, Aggarwal C, Wang S (2021) Nrgnn: Learning a label noise resistant graph neural network on sparsely and noisily labeled graphs. In: Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp 227–236

  24. Yu X, Han B, Yao J, Niu G, Tsang I, Sugiyama M (2019) How does disagreement help generalization against label corruption? In: International Conference on Machine Learning, pp 7164–7173. PMLR

  25. Nguyen DT, Mummadi CK, Ngo TPN, Nguyen THP, Beggel L, Brox T (2019) Self: Learning to filter noisy labels with self-ensembling. In: International Conference on Learning Representations, ICLR

  26. Ren M, Zeng W, Yang B, Urtasun R (2018) Learning to reweight examples for robust deep learning. In: International Conference on Machine Learning, pp 4334–4343. PMLR

  27. Arazo E, Ortego D, Albert P, O’Connor N, McGuinness K (2019) Unsupervised label noise modeling and loss correction. In: International Conference on Machine Learning, pp 312–321. PMLR

  28. Lukasik M, Bhojanapalli S, Menon A, Kumar S (2020) Does label smoothing mitigate label noise? In: International Conference on Machine Learning, pp 6448–6458. PMLR

  29. Xiao T, Xia T, Yang Y, Huang C, Wang X (2015) Learning from massive noisy labeled data for image classification. In: Proc IEEE Conf Comput Vis Pattern Recognit, pp 2691–2699

  30. Wu Z, Pan S, Chen F, Long G, Zhang C, Philip SY (2020) A comprehensive survey on graph neural networks. IEEE transactions on neural networks and learning systems 32(1):4–24

    Article  MathSciNet  Google Scholar 

  31. Sun L, Dou Y, Yang C, Zhang K, Wang J, Philip SY, He L, Li B (2022) Adversarial attack and defense on graph data: A survey. IEEE Trans Knowl Data Eng, 1–20

  32. Liu Y, Jin M, Pan S, Zhou C, Zheng Y, Xia F, Yu P (2022) Graph self-supervised learning: A survey. IEEE Trans Knowl Data Eng, 1–1

  33. Ding K, Xu Z, Tong H, Liu H (2022) Data augmentation for deep graph learning: A survey. ACM SIGKDD Explorations Newsletter 24(2):61–77

    Article  Google Scholar 

  34. Bruna J, Zaremba W, Szlam A, LeCun Y (2014) Spectral networks and deep locally connected networks on graphs. In: 2nd International Conference on Learning Representations, ICLR

  35. Defferrard M, Bresson X, Vandergheynst P (2016) Convolutional neural networks on graphs with fast localized spectral filtering. In: Advances in Neural Information Processing Systems, pp 3844–3852

  36. Wu F, Souza A, Zhang T, Fifty C, Yu T, Weinberger K (2019) Simplifying graph convolutional networks. In: International Conference on Machine Learning 97:6861–6871. PMLR, ???

  37. Jiang J, Ma J, Liu X (2020) Multilayer spectral-spatial graphs for label noisy robust hyperspectral image classification. IEEE Transactions on Neural Networks and Learning Systems 33(2):839–852

    Article  Google Scholar 

  38. Wang X, Zhang M (2022) How powerful are spectral graph neural networks. In: International Conference on Machine Learning, pp 23341–23362. PMLR

  39. Li Q, Han Z, Wu X-M (2018) Deeper insights into graph convolutional networks for semi-supervised learning. In: Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, pp 3538–3545

  40. Chen D, Lin Y, Li W, Li P, Zhou J, Sun X (2020) Measuring and relieving the over-smoothing problem for graph neural networks from the topological view. In: Proceedings of the Thirty-Fourth AAAI Conference on Artificial Intelligence, pp 3438–3445

  41. Wang Y, Wang Y, Yang J, Lin Z (2021) Dissecting the diffusion process in linear graph convolutional networks. In: Advances in Neural Information Processing Systems

  42. Jiang B, Zhang Z, Lin D, Tang J, Luo B (2019) Semi-supervised learning with graph learning-convolutional networks. In: Proc IEEE/CVF Conf Comput Vis Pattern Recognit, pp 11313–11320

  43. Chen Y, Wu L, Zaki M (2020) Iterative deep graph learning for graph neural networks: Better and robust node embeddings. In: Advances in Neural Information Processing Systems

  44. Fatemi B, Asri LE, Kazemi SM (2021) Slaps: Self-supervision improves structure learning for graph neural networks. In: Advances in Neural Information Processing Systems

  45. Zhang M, Hu L, Shi C, Wang X (2020) Adversarial label-flipping attack and defense for graph neural networks. In: 2020 IEEE International Conference on Data Mining (ICDM), pp 791–800. IEEE

  46. Li Y, Yin J, Chen L (2021) Unified robust training for graph neural networks against label noise. In: Pacific-Asia Conference on Knowledge Discovery and Data Mining, pp 528–540. Springer

  47. Kolesnikov A, Zhai X, Beyer L (2019) Revisiting self-supervised visual representation learning. In: Proc IEEE Conf Comput Vis Pattern Recognit, pp 1920–1929

  48. Hu Z, Dong Y, Wang K, ChangK-W, Sun Y (2020) Gpt-gnn: Generative pre-training of graph neural networks. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp 1857–1867

  49. You Y, Chen T, Wang Z, Shen Y (2020) When does self-supervision help graph convolutional networks? In: International Conference on Machine Learning, pp 10871–10880. PMLR

  50. Hassani K, Khasahmadi AH (2020) Contrastive multi-view representation learning on graphs. In: International Conference on Machine Learning, pp 4116–4126. PMLR

  51. Xu D, Cheng W, Luo D, Chen H, Zhang X (2021) Infogcl: Information-aware graph contrastive learning. Advances in Neural Information Processing Systems 34

  52. Li H, Wang X, Zhang Z, Yuan Z, Li H, Zhu W (2021) Disentangled contrastive learning on graphs. Advances in Neural Information Processing Systems 34

  53. Chen J, Gong Z, Mo J, Wang W, Wang C, Dong X, Liu W, Wu K (2021) Self-training enhanced: Network embedding and overlapping community detection with adversarial learning. IEEE Transactions on Neural Networks and Learning Systems 33(11):6737–6748

    Article  MathSciNet  Google Scholar 

  54. Lee N, Lee J, Park C (2022) Self-supervised graph representation learning via positive mining. Inf Sci 611:476–493

    Article  Google Scholar 

  55. Xu X, Deng C, Xie Y, Ji S (2023) Group contrastive self-supervised learning on graphs. IEEE Transactions on Pattern Analysis and Machine Intelligence 45(3):3169–3180

    Google Scholar 

  56. Xiao Y, Xing Z, Liu AX, Bai L, Pei Q, Yao L (2022) Cure-gnn: A robust curvature-enhanced graph neural network against adversarial attacks. IEEE Transactions on Dependable and Secure Computing, 1–16

  57. Li R, Wang S, Zhu F, Huang J (2018) Adaptive graph convolutional neural networks. In: Proceedings of the AAAI Conference on Artificial Intelligence 32

  58. Li J, Socher R, Hoi SC (2020) Dividemix: Learning with noisy labels as semi-supervised learning. In: International Conference on Learning Representations, ICLR

  59. Nguyen T, Mummadi C, Ngo T, Beggel L, Brox T (2020) Self: learning to filter noisy labels with self-ensembling. In: International Conference on Learning Representations, ICLR

  60. Zhou Z, Hu Y, Zhang Y, Chen J, Cai H (2022) Multiview deep graph infomax to achieve unsupervised graph embedding. IEEE Transactions on Cybernetics, 1–11

  61. Qiu J, Chen Q, Dong Y, Zhang J, Yang H, Ding M, Wang K, Tang J (2020) Gcc: Graph contrastive coding for graph neural network pre-training. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp 1150–1160

  62. Zhu Y, Xu Y, Yu F, Liu Q, Wu S, Wang L (2021) Graph contrastive learning with adaptive augmentation. Proceedings of the Web Conference 2021, pp 2069–2080

  63. Zhu D, Zhang Z, Cui P, Zhu W (2019) Robust graph convolutional networks against adversarial attacks. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp 1399–1407

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Technology, Nanjing University, No.163 Xianlin Avenue, Qixia District, Nanjing, 100190, China

    Jinliang Yuan, Hualei Yu, Meng Cao, Jianqing Song, Junyuan Xie & Chongjun Wang

Authors
  1. Jinliang Yuan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hualei Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Meng Cao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jianqing Song
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Junyuan Xie
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Chongjun Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Jinliang Yuan or Chongjun Wang.

Ethics declarations

Funding and/or Conflicts of interests/Competing interests

This paper is supported by the National Natural Science Foundation of China (Grant No. 62192783, U1811462), the Collaborative Innovation Center of Novel Software Technology and Industrialization at Nanjing University. All authors certify that they have no affiliations with or involvement in any organization or entity with any financial interest or non-financial interest in the subject matter or materials discussed in this manuscript.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yuan, J., Yu, H., Cao, M. et al. Self-supervised robust Graph Neural Networks against noisy graphs and noisy labels. Appl Intell 53, 25154–25170 (2023). https://doi.org/10.1007/s10489-023-04836-6

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-023-04836-6

Keywords

Search

Navigation