Nothing Special   »   [go: up one dir, main page]
Skip to main content

Advertisement

Springer Nature Link
Log in

Hypernetwork-driven centralized contrastive learning for federated graph classification

  • Published:
World Wide Web Aims and scope Submit manuscript

Abstract

In the domain of Graph Federated Learning (GFL), prevalent methods often focus on local client data, which can limit the understanding of broader global patterns and pose challenges with Non-IID (Non-Independent and Identically Distributed) issues in cross-domain datasets. Direct aggregation can lead to a reduction in the differences among various clients, which is detrimental to personalized datasets. Contrastive Learning (CL) has emerged as an effective tool for enhancing a model’s ability to distinguish variations across diverse views but has not been fully leveraged in GFL. This study introduces a novel hypernetwork-based method, termed CCL (Centralized Contrastive Learning), which is a server-centric innovation that effectively addresses the challenges posed by traditional client-centric approaches in heterogeneous datasets. CCL integrates global patterns from multiple clients, capturing a wider range of patterns and significantly improving GFL performance. Our extensive experiments, including both supervised and unsupervised scenarios, demonstrate CCL’s superiority over existing models, its remarkable compatibility with standard backbones, and its ability to enhance GFL performance across various settings.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Algorithm 1
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Data Availability

No datasets were generated or analysed during the current study.

References

  1. McMahan, B., Moore, E., Ramage, D., Hampson, S., Arcas, B.A.: Communication-efficient learning of deep networks from decentralized data. In: Artificial Intelligence and Statistics, pp. 1273–1282 (2017). PMLR

  2. Baek, J., Jeong, W., Jin, J., Yoon, J., Hwang, S.J.: Personalized subgraph federated learning. In: International Conference on Machine Learning, pp. 1396–1415 (2023). PMLR

  3. Tan, Y., Liu, Y., Long, G., Jiang, J., Lu, Q., Zhang, C.: Federated learning on non-iid graphs via structural knowledge sharing. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 37, pp. 9953–9961 (2023)

  4. Xie, H., Ma, J., Xiong, L., Yang, C.: Federated graph classification over non-iid graphs. Adv. Neural Inf. Proc. Syst. 34, 18839–18852 (2021)

    Google Scholar 

  5. He, K., Fan, H., Wu, Y., Xie, S., Girshick, R.: Momentum contrast for unsupervised visual representation learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9729–9738 (2020)

  6. Chen, T., Kornblith, S., Norouzi, M., Hinton, G.: A simple framework for contrastive learning of visual representations. In: International Conference on Machine Learning, pp. 1597–1607 (2020). PMLR

  7. You, Y., Chen, T., Sui, Y., Chen, T., Wang, Z., Shen, Y.: Graph contrastive learning with augmentations. Adv. Neural Inf. Process. Syst. 33, 5812–5823 (2020)

    Google Scholar 

  8. Zhu, Y., Xu, Y., Yu, F., Liu, Q., Wu, S., Wang, L.: Graph contrastive learning with adaptive augmentation. In: Proceedings of the Web Conference 2021, pp. 2069–2080 (2021)

  9. Li, Q., He, B., Song, D.: Model-contrastive federated learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10713–10722 (2021)

  10. Xia, J., Li, M., Tang, Y., Yang, S.: Course map learning with graph convolutional network based on aucm. World Wide Web 26(5), 3483–3502 (2023)

    Article  Google Scholar 

  11. Xu, D., Chen, Y., Cui, N., Li, J.: Towards multi-dimensional knowledge-aware approach for effective community detection in lbsn. World Wide Web 26(4), 1435–1458 (2023)

    Article  Google Scholar 

  12. Tang, Y., Li, J., Haldar, N.A.H., Guan, Z., Xu, J., Liu, C.: Reliability-driven local community search in dynamic networks. IEEE Transactions on Knowledge and Data Engineering (2023)

  13. Dong, S., Miao, X., Liu, P., Wang, X., Cui, B., Li, J.: Het-kg: Communication-efficient knowledge graph embedding training via hotness-aware cache. In: 2022 IEEE 38th International Conference on Data Engineering (ICDE), pp. 1754–1766 (2022). IEEE

  14. Cai, B., Xiang, Y., Gao, L., Zhang, H., Li, Y., Li, J.: Temporal knowledge graph completion: A survey. arXiv:2201.08236 (2022)

  15. Ye, R., Ni, Z., Wu, F., Chen, S., Wang, Y.: Personalized federated learning with inferred collaboration graphs. In: International Conference on Machine Learning, pp. 39801–39817(2023). PMLR

  16. Caldarola, D., Mancini, M., Galasso, F., Ciccone, M., Rodolà, E., Caputo, B.: Cluster-driven graph federated learning over multiple domains. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2749–2758 (2021)

  17. Yao, Y., Jin, W., Ravi, S., Joe-Wong, C.: Fedgcn: Convergence and communication tradeoffs in federated training of graph convolutional networks. arXiv:2201.12433 (2022)

  18. Yin, Y., Li, Y., Gao, H., Liang, T., Pan, Q.: Fgc: Gcn-based federated learning approach for trust industrial service recommendation. IEEE Transactions on Industrial Informatics 19(3), 3240–3250 (2022)

    Article  Google Scholar 

  19. Zhang, H., Koh, J.Y., Baldridge, J., Lee, H., Yang, Y.: Cross-modal contrastive learning for text-to-image generation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 833–842 (2021)

  20. Li, Q., Diao, Y., Chen, Q., He, B.: Federated learning on non-iid data silos: an experimental study. In: 2022 IEEE 38th International Conference on Data Engineering (ICDE), pp. 965–978 (2022). IEEE

  21. Fu, X., Zhang, B., Dong, Y., Chen, C., Li, J.: Federated graph machine learning: a survey of concepts, techniques, and applications. ACM SIGKDD Explorations Newsletter 24(2), 32–47 (2022)

    Article  Google Scholar 

  22. Wang, Z., Kuang, W., Xie, Y., Yao, L., Li, Y., Ding, B., Zhou, J.: Federatedscope-gnn: towards a unified, comprehensive and efficient package for federated graph learning. In: Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 4110–4120 (2022)

  23. Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv:1412.6980 (2014)

  24. Li, T., Sahu, A.K., Zaheer, M., Sanjabi, M., Talwalkar, A., Smith, V.: Federated optimization in heterogeneous networks. Proceedings of Machine Learning and Systems 2, 429–450 (2020)

    Google Scholar 

  25. Arivazhagan, M.G., Aggarwal, V., Singh, A.K., Choudhary, S.: Federated learning with personalization layers. arXiv:1912.00818 (2019)

  26. Zhang, K., Yang, C., Li, X., Sun, L., Yiu, S.M.: Subgraph federated learning with missing neighbor generation. Adv. Neural Inf. Proc. Sys. 34, 6671–6682 (2021)

    Google Scholar 

  27. Xu, K., Hu, W., Leskovec, J., Jegelka, S.: How powerful are graph neural networks? arXiv:1810.00826 (2018)

  28. O Pinheiro, P.O., Almahairi, A., Benmalek, R., Golemo, F., Courville, A.C.: Unsupervised learning of dense visual representations. Adv. Neural Inf. Proc. Syst. 33, 4489–4500 (2020)

  29. Haj-Yahia, Z., Sieg, A., Deleris, L.A.: Towards unsupervised text classification leveraging experts and word embeddings. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 371–379 (2019)

  30. Li, J., Li, Z., Mou, L., Jiang, X., Lyu, M., King, I.: Unsupervised text generation by learning from search. Adv. Neural Inf. Proc. Syst. 33, 10820–10831 (2020)

    Google Scholar 

  31. LeCun, Y., Boser, B., Denker, J.S., Henderson, D., Howard, R.E., Hubbard, W., Jackel, L.D.: Backpropagation applied to handwritten zip code recognition. Neural Comput. 1(4), 541–551 (1989)

    Article  Google Scholar 

  32. Kingma, D.P., Welling, M.: Auto-encoding variational bayes. arXiv:1312.6114 (2013)

Download references

Acknowledgements

We are very grateful to all those who helps me put these ideas and put them into practice.

Funding

This work is supported by National Natural Science Foundation of China under grants 62376103, 62302184, 62206102, and Science and Technology Support Program of Hubei Province under grant 2022BAA046.

Author information

Authors and Affiliations

  1. School of Computer Science & Technology, Huazhong University of Science and Technology, Guanshan Street, Hubei, 430074, Wuhan, China

    Jianian Zhu, Yichen Li, Haozhao Wang, Yining Qi & Ruixuan Li

Authors
  1. Jianian Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yichen Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Haozhao Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yining Qi
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ruixuan Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Jianian Zhu wrote the main manuscript text. Yichen Li, Haozhao Wang, Yining Qi and Ruixuan Li reviewed and revised this manuscript.

Corresponding authors

Correspondence to Yining Qi or Ruixuan Li.

Ethics declarations

Competing Interests

The authors declare no competing interests.

Ethical Approval

Not applicable.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhu, J., Li, Y., Wang, H. et al. Hypernetwork-driven centralized contrastive learning for federated graph classification. World Wide Web 27, 56 (2024). https://doi.org/10.1007/s11280-024-01292-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11280-024-01292-1

Keywords