Nothing Special   »   [go: up one dir, main page]
Skip to main content

Advertisement

Springer Nature Link
Log in

Robust graph convolutional networks with directional graph adversarial training

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Graph convolutional networks (GCNs), an emerging type of neural network model on graphs, have presented state-of-the-art performance on the node classification task. However, recent studies show that neural networks are vulnerable to the small but deliberate perturbations on input features. And GCNs could be more sensitive to the perturbations since the perturbations from neighbor nodes exacerbate the impact on a target node through the convolution. Adversarial training (AT) is a regularization technique that has been shown capable of improving the robustness of the model against perturbations on image classification. However, directly adopting AT on GCNs is less effective since AT regards examples as independent of each other and does not consider the impact from connected examples. In this work, we explore AT on graph and propose a graph-specific AT method, Directional Graph Adversarial Training (DGAT), which incorporates the graph structure into the adversarial process and automatically identifies the impact of perturbations from neighbor nodes. Concretely, we consider the impact from the connected nodes to define the neighbor perturbation which restricts the perturbation direction on node features towards their neighbor nodes, and additionally introduce an adversarial regularizer to defend the worst-case perturbations. In this way, DGAT can resist the impact of worst-case adversarial perturbations and reduce the impact of perturbations from neighbor nodes. Extensive experiments demonstrate that DGAT can effectively improve the robustness and generalization performance of GCNs. Specially, GCNs with DGAT can provide better performance when there are rare few labels available for training.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Li D, Wen G, Li X, Cai X (2019) Graph-based dynamic ensemble pruning for facial expression recognition. Appl Intell 49(9):3188–3206

    Article  Google Scholar 

  2. Fang Y, Wang H, Zhao L, Yu F, Wang C (2020) Dynamic knowledge graph based fake-review detection. Appl Intell: 1–15

  3. Wu Z, Pan S, Chen F, Long G, Zhang C, Philip SY (2020) A comprehensive survey on graph neural networks. In: IEEE transactions on neural networks and learning systems

  4. Bruna J, Zaremba W, Szlam A, LeCun Y (2013) Spectral networks and locally connected networks on graphs, arXiv:1312.6203

  5. Henaff M, Bruna J, LeCun Y (2015) Deep convolutional networks on graph-structured data, arXiv:1506.05163

  6. Defferrard M, Bresson X, Vandergheynst P (2016) Convolutional neural networks on graphs with fast localized spectral filtering. In: Advances in neural information processing systems, pp 3844–3852

  7. Kipf TN, Welling M (2016) Semi-supervised classification with graph convolutional networks, arXiv:1609.02907

  8. Wu F, Zhang T, Souza AHd Jr, Fifty C, Yu T, Weinberger KQ (2019) Simplifying graph convolutional networks, arXiv:1902.07153

  9. Gao H, Wang Z, Ji S (2018) Large-scale learnable graph convolutional networks. In: Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining. ACM, pp 1416–1424

  10. Ma J, Cui P, Kuang K, Wang X, Zhu W (2019) Disentangled graph convolutional networks. In: International conference on machine learning, pp 4212–4221

  11. Wang X, Zhu M, Bo D, Cui P, Shi C, Pei J (2020) Am-gcn: Adaptive multi-channel graph convolutional networks. In: Proceedings of the 26th ACM SIGKDD International conference on knowledge discovery & data mining, pp 1243–1253

  12. Yao L, Mao C, Luo Y (2019) Graph convolutional networks for text classification. Proceedings of the AAAI Conference on Artificial Intelligence 33:7370–7377

    Article  Google Scholar 

  13. Shi L, Zhang Y, Cheng J, Lu H (2019) Two-stream adaptive graph convolutional networks for skeleton-based action recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 12026–12035

  14. Ying R, He R, Chen K, Eksombatchai P, Hamilton WL, Leskovec J (2018) Graph convolutional neural networks for web-scale recommender systems. In: Proceedings of the 24th ACM SIGKDD International conference on knowledge discovery & data mining. ACM, pp 974–983

  15. He X, Deng K, Wang X, Li Y, Zhang Y, Wang M (2020) Lightgcn: Simplifying and powering graph convolution network for recommendation, arXiv:2002.02126

  16. Szegedy C, Zaremba W, Sutskever I, Bruna J, Erhan D, Goodfellow I, Fergus R (2013) Intriguing properties of neural networks, arXiv:1312.6199

  17. Xu H, Ma Y, Liu H-C, Deb D, Liu H, Tang J-L, Jain AK (2020) Adversarial attacks and defenses in images, graphs and text: A review. Int J Autom Comput 17(2):151–178

    Article  Google Scholar 

  18. Zügner D., Akbarnejad A, Günnemann S. (2018) Adversarial attacks on neural networks for graph data. In: Proceedings of the 24th ACM SIGKDD International conference on knowledge discovery & data mining. ACM, pp 2847–2856

  19. Dai H, Li H, Tian T, Huang X, Wang L, Zhu J, Song L (2018) Adversarial attack on graph structured data: 1115–1124. arXiv:1806.02371

  20. Zügner D., Günnemann S. (2019) Adversarial attacks on graph neural networks via meta learning, arXiv:1902.08412

  21. Chen L, Li J, Peng J, Xie T, Cao Z, Xu K, He X, Zheng Z (2020) A survey of adversarial learning on graphs, arXiv:2003.05730

  22. Goodfellow IJ, Shlens J, Szegedy C (2014) Explaining and harnessing adversarial examples, arXiv:1412.6572

  23. Miyato T, Maeda S, Koyama M, Ishii S (2018) Virtual adversarial training: a regularization method for supervised and semi-supervised learning. IEEE Trans Pattern Anal Mach Intell 41(8):1979–1993

    Article  Google Scholar 

  24. Madry A, Makelov A, Schmidt L, Tsipras D, Vladu A (2017) Towards deep learning models resistant to adversarial attacks, arXiv:1706.06083

  25. Miyato T, Dai AM, Goodfellow I (2016) Adversarial training methods for semi-supervised text classification, arXiv:1605.07725

  26. He X, He Z, Du X, Chua T-S (2018) Adversarial personalized ranking for recommendation. In: The 41st International ACM SIGIR conference on research & development in information retrieval. ACM, pp 355–364

  27. Sun K, Lin Z, Guo H, Zhu Z (2019) Virtual adversarial training on graph convolutional networks in node classification. In: Chinese conference on pattern recognition and computer vision (PRCV). Springer, pp 431–443

  28. Feng F, He X, Tang J, Chua T. -S. (2019) Graph adversarial training: Dynamically regularizing based on graph structure. IEEE Transactions on knowledge and data engineering

  29. Shuman DI, Narang SK, Frossard P, Ortega A, Vandergheynst P (2012) The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains, arXiv:1211.0053

  30. Hamilton W, Ying Z, Leskovec J (2017) Inductive representation learning on large graphs. In: Advances in neural information processing systems, pp 1024–1034

  31. Veličković P., Cucurull G, Casanova A, Romero A, Lio P, Bengio Y (2017) Graph attention networks, arXiv:1710.10903

  32. Xu K, Li C, Tian Y, Sonobe T, Kawarabayashi K, Jegelka S (2018) Representation learning on graphs with jumping knowledge networks, arXiv:1806.03536

  33. Liu M, Gao H, Ji S (2020) Towards deeper graph neural networks. In: Proceedings of the 26th ACM SIGKDD International conference on knowledge discovery & data mining, pp 338–348

  34. Chang H, Rong Y, Xu T, Huang W, Zhang H, Cui P, Zhu W, Huang J (2020) A restricted black-box adversarial framework towards attacking graph embedding models. In: AAAI, pp 3389–3396

  35. Zhang Y, Pal S, Coates M, Ustebay D (2019) Bayesian graph convolutional neural networks for semi-supervised classification. In: Proceedings of the AAAI conference on artificial intelligence, vol 33, pp 5829–5836

  36. Zhu D, Zhang Z, Cui P, Zhu W (2019) Robust graph convolutional networks against adversarial attacks. In: Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, pp 1399–1407

  37. Dai Q, Shen X, Zhang L, Li Q, Wang D (2019) Adversarial training methods for network embedding. In: The World Wide Web conference. ACM, pp 329–339

  38. Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets. In: Advances in neural information processing systems, pp 2672–2680

  39. Wang K, Gou C, Duan Y, Lin Y, Zheng X, Wang F. -Y. (2017) Generative adversarial networks: introduction and outlook. IEEE/CAA Journal of Automatica Sinica 4(4):588–598

    Article  MathSciNet  Google Scholar 

  40. Wang H, Wang J, Wang J, Zhao M, Zhang W, Zhang F, Xie X, Guo M (2018) Graphgan: Graph representation learning with generative adversarial nets. In: Thirty-second AAAI conference on artificial intelligence

  41. Ding M, Tang J, Zhang J (2018) Semi-supervised learning on graphs with generative adversarial nets. In: Proceedings of the 27th ACM international conference on information and knowledge management, pp 913–922

  42. Dai Q, Li Q, Tang J, Wang D (2018) Adversarial network embedding. In: Thirty-Second AAAI conference on artificial intelligence

  43. Yu W, Zheng C, Cheng W, Aggarwal CC, Song D, Zong B, Chen H, Wang W (2018) Learning deep network representations with adversarially regularized autoencoders. In: Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining, pp 2663–2671

  44. Pan S, Hu R, Long G, Jiang J, Yao L, Zhang C (2018) Adversarially regularized graph autoencoder for graph embedding, arXiv:1802.04407

  45. Chen J, Zhu J, Song L (2017) Stochastic training of graph convolutional networks with variance reduction, arXiv:1710.10568

  46. Kurakin A, Goodfellow I, Bengio S (2016) Adversarial machine learning at scale, arXiv:1611.01236

  47. Yang Z, Cohen W, Salakhudinov R (2016) Revisiting semi-supervised learning with graph embeddings. In: International conference on machine learning. PMLR, pp 40–48

  48. Zhu X, Ghahramani Z, Lafferty JD (2003) Semi-supervised learning using gaussian fields and harmonic functions. In: Proceedings of the 20th International conference on Machine learning (ICML-03), pp 912–919

  49. Perozzi B, Al-Rfou R, Skiena S (2014) Deepwalk: Online learning of social representations. In: Proceedings of the 20th ACM SIGKDD international conference on knowledge discovery and data mining. ACM, pp 701–710

  50. Weston J, Ratle F, Mobahi H, Collobert R (2012) Deep learning via semi-supervised embedding. In: Neural networks: tricks of the trade. Springer, pp 639–655

  51. Ibrahim NMA, Chen L (2015) Link prediction in dynamic social networks by integrating different types of information. Appl Intell 42(4):738–750

    Article  Google Scholar 

Download references

Acknowledgments

The work described in this paper was supported by the Key-Area Research and Development Program of Guangdong Province (No. 2018B010109001), the National Natural Science Foundation of China (No. 11801595), the Guangdong Basic and Applied Basic Research Foundation (No. 2019A1515011043), the Natural Science Foundation of Guangdong (No. 2018A030310076) and the CCF-Tencent Open Fund WeBank Special Funding.

Author information

Authors and Affiliations

  1. School of Computer Science and Engineering, Sun Yat-sen University, Guangzhou, China

    Weibo Hu, Chuan Chen, Yaomin Chang, Zibin Zheng & Yunfei Du

Authors
  1. Weibo Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chuan Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yaomin Chang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zibin Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yunfei Du
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chuan Chen.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hu, W., Chen, C., Chang, Y. et al. Robust graph convolutional networks with directional graph adversarial training. Appl Intell 51, 7812–7826 (2021). https://doi.org/10.1007/s10489-021-02272-y

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-021-02272-y

Keywords

Search

Navigation