Toward Robust and Generalizable Federated Graph Neural Networks for Decentralized Spatial-Temporal Data Modeling
Authors: 
Yuxing Tian, Lei Liu, Jie Feng, Qingqi Pei, Chen Chen, Jun Du, Celimuge WuAuthors Info & Claims
Pages 2637 - 2650
Abstract
Federated learning has been combined with graph learning for modeling spatial-temporal data while maintaining data confidentiality and safety. However, there are still several issues: 1) In practical usage, some clients may be unable to participate in the model inference due to poor network signal, malicious attacks, etc. 2) In the communication process, the uploaded information is easily disturbed by noise. The performance of the graph model will be seriously affected by its low robustness. Additionally, the assumption of identical distribution between the training and testing domain does not hold in practical scenarios, resulting in overfitting and poor generalization ability of the trained models. 3) The relations that exist among clients may change dynamically over time and manually constructing the graph structure of clients may not accurately represent the relations among clients. In this paper, we address all the above limitations by proposing a robust hierarchical split-federated graph model named DCSFG. Specifically, DCSFG combines split-federated learning and spatial-temporal graph model to better capture the spatial-temporal dependencies. We propose a Dropclient method and introduce the uncertainty estimation to enhance the robustness and generlization ability of the model. We also design a dual-sub-decoders structure for clients so that they can perform predictions locally and independently when they are unable to participate in the inference process. A novel hierarchical graph message passing structure is proposed to enable each client to perceive the global and local information. The extensive experimental results demonstrate the effectiveness of DCSFG.
References
[1]
L. Liu, J. Feng, X. Mu, Q. Pei, D. Lan, and M. Xiao, “Asynchronous deep reinforcement learning for collaborative task computing and on-demand resource allocation in vehicular edge computing,” IEEE Trans. Intell. Transp. Syst., vol. 24, no. 12, pp. 15513–15526, Dec. 2023.
[2]
R. Ying, R. He, K. Chen, P. Eksombatchai, W. Hamilton, and J. Leskovec, “Graph convolutional neural networks for Web-scale recommender systems,” in Proc. 24th ACM SIGKDD Int. Conf. Knowl. Discovery Data Min., 2018, pp. 974–983.
[3]
C. Gao, X. Wang, X. He, and Y. Li, “Graph neural networks for recommender system,” in Proc. WSDM, 2022, pp. 1623–1625.
[4]
X. Zhanget al., “Traffic flow forecasting with spatial-temporal graph diffusion network,” in Proc. AAAI Conf. Artif. Intell., 2021, pp. 15008–15015.
[5]
M. Li and Z. Zhu, “Spatial-temporal fusion graph neural networks for traffic flow forecasting,” in Proc. AAAI Conf. Artif. Intell., 2021, pp. 4189–4196.
[6]
Z. Shaoet al., “Decoupled dynamic spatial-temporal graph neural network for traffic forecasting,” Proc. VLDB Endowment, vol. 15, no. 11, pp. 2733–2746, 2022.
[7]
S. Lan, Y. Ma, W. Huang, W. Wang, H. Yang, and P. Li, “DSTAGNN: Dynamic spatial-temporal aware graph neural network for traffic flow forecasting,” in Proc. ICML, 2022, pp. 11906–11917.
[8]
P. Voigt and A. Bussche, The EU General Data Protection Regulation (GDPR): A Practical Guide. Cham, Switzerland: Springer, 2017.
[9]
H. B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas, “Communication-efficient learning of deep networks from decentralized data,” in Proc. AISTATS, 2016, pp. 1–10.
[10]
J. Konečný, B. McMahan, and D. Ramage, “Federated optimization: Distributed optimization beyond the datacenter,” 2015, arXiv:1511.03575.
[11]
X. Fu, B. Zhang, Y. Dong, C. Chen, and J. Li, “Federated graph machine learning: A survey of concepts, techniques, and applications,” 2022, arXiv.2207.11812.
[12]
F. Chen, G. Long, Z. Wu, T. Zhou, and J. Jiang, “Personalized federated learning with a graph,” in Proc. IJCAI, 2022, pp. 1–8.
[13]
P. Xing, S. Lu, L. Wu, and H. Yu, “BiG-fed: Bilevel optimization enhanced graph-aided federated learning,” IEEE Trans. Big Data, early access, Jul. 18, 2023. 10.1109/TBDATA.2022.3191439.
[14]
L. Liuet al., “Multilevel federated learning-based intelligent traffic flow forecasting for transportation network management,” IEEE Trans. Netw. Service Manag., vol. 20, no. 2, pp. 1446–1458, Jun. 2023.
[15]
C. Meng, S. Rambhatla, and Y. Liu, “Cross-node federated graph neural network for spatio-temporal data modeling,” in Proc. KDD, 2021, pp. 14–18.
[16]
P. W. Battagliaet al., “Relational inductive biases, deep learning, and graph networks,” 2018, arXiv:1806.01261.
[17]
B. Yu, H. Yin, and Z. Zhu, “Spatio-temporal graph convolutional networks: A deep learning framework for traffic forecasting,” in Proc. IJCAI, 2018, pp. 1–7.
[18]
C. Zhang, S. Zhang, J. J. Q. Yu, and S. Yu, “FASTGNN: A topological information protected federated learning approach for traffic speed forecasting,” IEEE Trans. Ind. Informat., vol. 17, no. 12, pp. 8464–8474, Dec. 2021.
[19]
T. N. Kipf and M. Welling, “Semi-supervised classification with graph convolutional networks,” 2016, arXiv:1609.02907.
[20]
Y. Wang, W. Wang, Y. Liang, Y. Cai, J. Liu, and B. Hooi, “NodeAug: Semi-supervised node classification with data augmentation,” in Proc. 26th ACM SIGKDD Int. Conf. Knowl. Discovery Data Min., 2020, pp. 207–217.
[21]
L. Wu, H. Lin, Z. Gao, C. Tan, and S. Z. Li. “Graphmixup: Improving class-imbalanced node classification on graphs by self-supervised context prediction,” 2021, arXiv:2106.11133.
[22]
L. Luo, R. Haffari, and S. Pan, “Graph sequential neural ode process for link prediction on dynamic and sparse graphs,” in Proc. 16th ACM Int. Conf. Web Search Data Min., 2022, pp. 778–786.
[23]
X. Changet al., “Continuous-time dynamic graph learning via neural interaction processes,” in Proc. 29th ACM Int. Conf. Inf. Knowl. Manag., 2020, pp. 145–154.
[24]
H. Penget al., “Dynamic graph convolutional network for long-term traffic flow prediction with reinforcement learning,” Inf. Sci., vol. 578, pp. 401–416, Nov. 2021.
[25]
W. Shaoet al., “Long-term spatio-temporal forecasting via dynamic multiple-graph attention,” 2022, arXiv:2204.11008.
[26]
O. Gupta and R. Raskar, “Distributed learning of deep neural network over multiple agents,” 2018, arXiv:1810.06060.
[27]
C. Thapa, M. A. P. Chamikara, S. Camtepe, and L. Sun, “SplitFed: When federated learning meets split learning,” in Proc. AAAI Conf. Artif. Intell., 2022, pp. 8485–8493.
[28]
Z. Yanget al., “Robust split federated learning for u-shaped medical image networks,” 2022, arXiv:2212.06378.
[29]
Q. Duan, S. Hu, R. Deng, and Z. Lu, “Combined federated and split learning in edge computing for ubiquitous Intelligence in Internet of Things: State-of-the-art and future directions,” Sensors, vol. 22, no. 16, p. 5983, 2022.
[30]
K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proc. CVPR, 2015, pp. 1–9.
[31]
Y. Rong, W. Huang, T. Xu, and J. Huang, “Dropedge: Towards deep graph convolutional networks on node classification,” in Proc. ICLR, 2020, pp. 1–17.
[32]
W. L. Hamilton, R. Ying, and J. Leskovec, “Inductive representation learning on large graphs,” 2017, arXiv:1706.02216.
[33]
J. Chen, T. Ma, and C. Xiao, “FastGCN: Fast learning with graph convolutional networks via importance sampling,” in Proc. Int. Conf. Learn. Represent., 2018, pp. 1–15.
[34]
C. Zhang, Q. Li, and D. Song, “Aspect-based sentiment classification with aspect-specific graph convolutional networks.” in Proc. 9th Int. Joint Conf. Empir. Methods Nat. Lang. Process. (EMNLP-IJCNLP), 2019, pp. 4560–4570.
[35]
Z. Houet al., “GraphMAE: Self-supervised masked graph autoencoders,” in Proc. KDD, 2022, pp. 1–11.
[36]
Y. Li, R. Yu, C. Shahabi, and Y. Liu, “Graph convolutional recurrent neural network: Data-driven traffic forecasting,” in Proc. ICLR, 2018, pp. 1–16.
Recommendations
Self-supervised robust Graph Neural Networks against noisy graphs and noisy labels
AbstractIn the paper, we first explore a novel problem of training the robust Graph Neural Networks (GNNs) against noisy graphs and noisy labels. To the problem, we propose a general Self-supervised Robust Graph Neural Network framework that consists of ...
Towards Robust Graph Neural Networks for Noisy Graphs with Sparse Labels
WSDM '22: Proceedings of the Fifteenth ACM International Conference on Web Search and Data MiningGraph Neural Networks (GNNs) have shown their great ability in modeling graph structured data. However, real-world graphs usually contain structure noises and have limited labeled nodes. The performance of GNNs would drop significantly when trained on ...
Robust Training of Graph Neural Networks via Noise Governance
WSDM '23: Proceedings of the Sixteenth ACM International Conference on Web Search and Data MiningGraph Neural Networks (GNNs) have become widely-used models for semi-supervised learning. However, the robustness of GNNs in the presence of label noise remains a largely under-explored problem. In this paper, we consider an important yet challenging ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
1932-4537 © 2024 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See https://www.ieee.org/publications/rights/index.html for more information.
Publisher
IEEE Press
Publication History
Published: 10 April 2024
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 16 Dec 2024