Nothing Special   »   [go: up one dir, main page]
Skip to main content
Springer Nature Link
Log in

Graph contrastive learning with cross-encoder for community discovery

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Community detection is a crucial task that enables the extraction of valuable knowledge and patterns from complex networks. However, the node labels are often unavailable in most real-world applications, which poses limitations on practical implementations. It is common to deal with graphs that have side information. How to effectively fuse the topology information of a graph and the attribute information of nodes also brings great challenges to community discovery task. In this paper, a graph contrastive learning algorithm with cross-encoder is proposed for community discovery. The proposed algorithm introduces graph convolution neural network and graph attention network to fuse the topology information and the attribute information of nodes. Then it employs cross-encoder to obtain the embedding vectors of nodes and the neural networks are trained by contrastive learning which can learn the embedding vectors of nodes from different views. The proposed algorithm and the comparison algorithms are conducted on multiple real data sets. The experimental results show that the proposed algorithm outperforms the comparison algorithms on the most experimental data sets. The experimental results prove that the proposed algorithm is effective for community discovery.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Notes

  1. https://scikit-learn.org/stable/modules/generated/

  2. https://github.com/bdy9527/SDCN

  3. https://github.com/PetarV-/DGI

  4. https://docs.dgl.ai/api/python/nn-pytorch.html

  5. https://github.com/shenweichen/GraphEmbedding

  6. https://github.com/zpeng27/GMI

  7. https://scikit-learn.org/stable/modules/generated/sklearn.neighbors.KNeighborsClassifier.html#sklearn.neighbors.KNeighborsClassifier

  8. https://scikit-learn.org/stable/modules/classes.html#sklearn-metrics-metrics

References

  1. Ando R, Zhang T (2006) Learning on graph with laplacian regularization. Adv Neural Inf Process Syst 19:25–32

    Google Scholar 

  2. Belkin M, Niyogi P, Sindhwani V (2006) Manifold regularization: A geometric framework for learning from labeled and unlabeled examples. J Mach Learn Res 7(11):2399–2434

    MathSciNet  Google Scholar 

  3. Bo D, Wang X, Shi C, Zhu M, Lu E, Cui P (2020) Structural deep clustering network. In: Proceedings of the web conference 2020, pp 1400–1410

  4. Brody S, Alon U, Yahav E (2021) How attentive are graph attention networks? In: International conference on learning representations

  5. Cai H, Zheng VW, Chang KCC (2018) A comprehensive survey of graph embedding: Problems, techniques, and applications. IEEE Trans Knowl Data Eng 30(9):1616–1637

    Article  Google Scholar 

  6. Chen C, Zhang M, Zhang Y, Ma W, Liu Y, Ma S (2020) Efficient heterogeneous collaborative filtering without negative sampling for recommendation. In: Proceedings of the AAAI conference on artificial intelligence, vol 34, pp 19–26

  7. Chen T, Kornblith S, Norouzi M, Hinton G (2020) A simple framework for contrastive learning of visual representations. In: International conference on machine learning, pp 1597–1607

  8. Corso G, Cavalleri L, Beaini D, Liò P, Veličković P (2020) Principal neighbourhood aggregation for graph nets. Adv Neural Inf Process Syst 33:13260–13271

    Google Scholar 

  9. Deng Z, Choi KS, Chung FL, Wang S (2010) Enhanced soft subspace clustering integrating within-cluster and between-cluster information. Pattern Recogn 43(3):767–781

    Article  ADS  Google Scholar 

  10. Girvan M, Newman ME (2002) Community structure in social and biological networks. Proc Natl Acad Sci 99(12):7821–7826

    Article  ADS  MathSciNet  CAS  PubMed  PubMed Central  Google Scholar 

  11. Goyal P, Ferrara E (2018) Graph embedding techniques, applications, and performance: A survey. Knowl-Based Syst 151:78–94

    Article  Google Scholar 

  12. Grill JB, Strub F, Altché F, Tallec C, Richemond PH, Buchatskaya E, Doersch C, Pires BA, Guo ZD, Azar MG, et al (2020) Bootstrap your own latent: A new approach to self-supervised learning. In: Advances in neural information processing systems

  13. Gui Q, Deng R, Xue P, Cheng X (2018) A community discovery algorithm based on boundary nodes and label propagation. Pattern Recogn Lett 109:103–109

    Article  ADS  Google Scholar 

  14. Hamilton WL, Ying R, Leskovec J (2017) Inductive representation learning on large graphs. In: Advances in neural information processing systems, pp 1024–1034

  15. He C, Zheng Y, Cheng J, Tang Y, Chen G, Liu H (2022) Semi-supervised overlapping community detection in attributed graph with graph convolutional autoencoder. Inf Sci 608:1464–1479

    Article  Google Scholar 

  16. He K, Fan H, Wu Y, Xie S, Girshick R (2020) Momentum contrast for unsupervised visual representation learning. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 9729–9738

  17. Hu F, Zhu Y, Wu S, Huang W, Wang L, Tan T (2021) Graphair: Graph representation learning with neighborhood aggregation and interaction. Pattern Recogn 112:107745

    Article  Google Scholar 

  18. Jin D, Yu Z, Jiao P, Pan S, He D, Wu J, Yu P, Zhang W (2021) A survey of community detection approaches: From statistical modeling to deep learning. IEEE Trans Knowl Data Eng

  19. Khandekar R, Kortsarz G, Mirrokni V (2014) On the advantage of overlapping clusters for minimizing conductance. Algorithmica 69(4):844–863

    Article  MathSciNet  Google Scholar 

  20. Kipf TN, Welling M (2016) Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907

  21. Kipf TN, Welling M (2017) Semi-supervised classification with graph convolutional networks. In: International conference on learning representations

  22. Lee DD, Seung HS (1999) Learning the parts of objects by non-negative matrix factorization. Nature 401(6755):788–791

    Article  ADS  CAS  PubMed  Google Scholar 

  23. Lee H, Lee J, Ng JYH, Natsev P (2020) Large scale video representation learning via relational graph clustering. In: Proceedings of the IEEE/CVF Conference on computer vision and pattern recognition, pp 6807–6816

  24. Li X, Jia M, Islam MT, Yu L, Xing L (2020) Self-supervised feature learning via exploiting multi-modal data for retinal disease diagnosis. IEEE Trans Med Imaging 39(12):4023–4033

    Article  PubMed  Google Scholar 

  25. Liu F, Xue S, Wu J, Zhou C, Hu W, Paris C, Nepal S, Yang J, Yu PS (2020) Deep learning for community detection: progress, challenges and opportunities. In: Proceedings of the 29th international joint conference on artificial intelligence, pp 4981– 4987

  26. Liu H, Fen L, Jian J, Chen L (2018) Overlapping community discovery algorithm based on hierarchical agglomerative clustering. Int J Pattern Recognit Artif Intell 32(03):1850008

    Article  MathSciNet  Google Scholar 

  27. Liu X, Zhang F, Hou Z, Mian L, Wang Z, Zhang J, Tang J (2020) Network representation learning: A macro and micro outlook. AI Open 1:57–81

    Google Scholar 

  28. Liu X, Zhang F, Hou Z, Wang Z, Mian L, Zhang J, Tang J (2021) Self-supervised learning: Generative or contrastive. IEEE Trans Knowl Data Eng

  29. Lu Z, Sun X, Wen Y, Cao G, La Porta T (2014) Algorithms and applications for community detection in weighted networks. IEEE Trans Parallel Distrib Syst 26(11):2916–2926

    Article  Google Scholar 

  30. Lv Z, Zhang S, Xiu W (2020) Solving the security problem of intelligent transportation system with deep learning. IEEE Trans Intell Transp Syst. https://doi.org/10.1109/TITS.2020.2980864

    Article  Google Scholar 

  31. Maas AL, Hannun AY, Ng AY (2013) Rectifier nonlinearities improve neural network acoustic models. In: Proceedings of the 30th international conference on machine learning, pp 3–8. Journal of Machine Learning Research

  32. Maćkiewicz A, Ratajczak W (1993) Principal components analysis (pca). Computers & Geosciences 19(3):303–342

    Article  ADS  Google Scholar 

  33. Mikolov T, Chen K, Corrado G, Dean J (2013) Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781

  34. Newman ME (2004) Fast algorithm for detecting community structure in networks. Phys Rev E 69(6):066133

    Article  ADS  CAS  Google Scholar 

  35. Newman ME (2006) Modularity and community structure in networks. Proc Natl Acad Sci 103(23):8577–8582

    Article  ADS  CAS  PubMed  PubMed Central  Google Scholar 

  36. Park N, Rossi R, Koh E, Burhanuddin IA, Kim S, Du F, Ahmed N, Faloutsos C (2022) Cgc: Contrastive graph clustering for community detection and tracking. In: Proceedings of the ACM web conference 2022, pp 1115–1126

  37. Peng Z, Huang W, Luo M, Zheng Q, Rong Y, Xu T, Huang J (2020) Graph representation learning via graphical mutual information maximization. In: Proceedings of The Web Conference 2020, pp 259–270

  38. Perozzi B, Al-Rfou R, Skiena S (2014) Deepwalk: Online learning of social representations. In: Proceedings of the 20th ACM SIGKDD international conference on knowledge discovery and data mining, pp 701–710

  39. Qiu J, Dong Y, Ma H, Li J, Wang C, Wang K, Tang J (2019) Netsmf: Large-scale network embedding as sparse matrix factorization. In: The World Wide Web Conference, pp. 1509–1520

  40. Qiu J, Dong Y, Ma H, Li J, Wang K, Tang J (2018) Network embedding as matrix factorization: Unifying deepwalk, line, pte, and node2vec. In: Proceedings of the eleventh ACM international conference on web search and data mining, pp 459–467

  41. Rong Y, Xu T, Huang J, Huang W, Cheng H, Ma Y, Wang Y, Derr T, Wu L, Ma T (2020) Deep graph learning: Foundations, advances and applications. In: Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining, pp 3555–3556

  42. Rossi RA, Ahmed NK, Koh E, Kim S, Rao A, Abbasi-Yadkori Y (2020) A structural graph representation learning framework. In: Proceedings of the 13th international conference on web search and data mining, pp 483–491

  43. Roweis ST, Saul LK (2000) Nonlinear dimensionality reduction by locally linear embedding. Science 290(5500):2323–2326

  44. Su X, Xue S, Liu F, Wu J, Yang J, Zhou C, Hu W, Paris C, Nepal S, Jin D, et al (2022) A comprehensive survey on community detection with deep learning. IEEE Trans Neural Netw Learn Syst

  45. Sun FY, Hoffmann J, Verma V, Tang J (2020) Infograph: Unsupervised and semi-supervised graph-level representation learning via mutual information maximization. In: International conference on learning representations

  46. Tang J, Qu M, Wang M, Zhang M, Yan J, Mei Q (2015) Line: Large-scale information network embedding. In: Proceedings of the 24th international conference on world wide web, pp 1067–1077

  47. Veličković P, Cucurull G, Casanova A, Romero A, Lio P, Bengio Y (2018) Graph attention networks. In: International conference on learning representations

  48. Velickovic P, Fedus W, Hamilton WL, Liò P, Bengio Y, Hjelm RD (2019) Deep graph infomax. In: International conference on learning representations

  49. Wang C, Pan S, Celina PY, Hu R, Long G, Zhang C (2022) Deep neighbor-aware embedding for node clustering in attributed graphs. Pattern Recogn 122:108230

    Article  Google Scholar 

  50. Wang M, Zuo W, Wang Y (2016) An improved density peaks-based clustering method for social circle discovery in social networks. Neurocomputing 179:219–227

    Article  Google Scholar 

  51. Wang X, Bo D, Shi C, Fan S , Ye Y, Yu PS (2020) A survey on heterogeneous graph embedding: Methods, techniques, applications and sources. arXiv:2011.14867

  52. Wang X, Qi GJ (2021) Contrastive learning with stronger augmentations. arXiv:2104.07713

  53. Wang Y, Min Y, Chen X, Wu J (2021) Multi-view graph contrastive representation learning for drug-drug interaction prediction. In: Proceedings of the web conference 2021, pp 2921–2933

  54. Wu X, Kumar V, Quinlan JR, Ghosh J, Yang Q, Motoda H, McLachlan GJ, Ng A, Liu B, Philip SY et al (2008) Top 10 algorithms in data mining. Knowl Inf Syst 14(1):1–37

    Article  Google Scholar 

  55. Xie T, Wang B, Kuo CCJ (2022) Graphhop: An enhanced label propagation method for node classification. IEEE Trans Neural Netw Learn Syst

  56. Xing S, Shan X, Fanzhen L, Jia W, Jian Y, Chuan Z, Wenbin H, Cecile P, Surya N, Di J et al (2022) A comprehensive survey on community detection with deep learning. IEEE Trans Neural Netw Learn Syst. https://doi.org/10.1109/TNNLS.2021.3137396

    Article  PubMed  Google Scholar 

  57. Xu S, Liu S, Feng L (2020) Manifold graph embedding with structure information propagation for community discovery. Knowl-Based Syst 208:106448

    Article  Google Scholar 

  58. Yang Z, Ding M, Zhou C, Yang H, Zhou J, Tang, J (2020) Understanding negative sampling in graph representation learning. In: Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining, pp 1666–1676

  59. You Y, Chen T, Sui Y, Chen T, Wang Z, Shen Y (2020) Graph contrastive learning with augmentations. In: Advances in neural information processing systems

  60. Zhang H, Lyu MR, King I (2015) Exploiting k-degree locality to improve overlapping community detection. In: Twenty-fourth international joint conference on artificial intelligence, pp 2394–2400

  61. Zhang X, Xu S, Lin W, Wang S (2023) Constrained social community recommendation. In: Proceedings of the 29th ACM SIGKDD conference on knowledge discovery and data mining, pp 5586–5596

  62. Zhou J, Cui G, Hu S, Zhang Z, Yang C, Liu Z, Wang L, Li C, Sun M (2020) Graph neural networks: A review of methods and applications. AI Open 1:57–81

    Article  Google Scholar 

  63. Zhu Y, Xu Y, Yu F, Liu Q, Wu S, Wang L (2021) Graph contrastive learning with adaptive augmentation. In: Proceedings of the web conference 2021, pp 2069–2080

Download references

Acknowledgements

This work was supported in part by the National Science Research Project of Department of Education in Guizhou Province Grant No. \(\left( \text {Grant No. KY}\left[ 2020 \right] 062 \right) \), in part by the National Natural Science Foundation of China under Grant 61672439, in part by the Fundamental Research Funds for the Central Universities under Grant 20720181004.

Author information

Authors and Affiliations

  1. School of Informatics Xiamen University, Xiamen, 361005, China

    Zhenpei Shan, Defu Zhang & Yunqi Lei

  2. School of Mathematics and Statistics, Liupanshui Normal University, Liupanshui, 553004, China

    Zhenpei Shan

Authors
  1. Zhenpei Shan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Defu Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yunqi Lei
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Defu Zhang.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Shan, Z., Zhang, D. & Lei, Y. Graph contrastive learning with cross-encoder for community discovery. Appl Intell 54, 2211–2224 (2024). https://doi.org/10.1007/s10489-024-05287-3

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-024-05287-3

Keywords

Search

Navigation