Abstract
Generative graph self-supervised learning (SSL), represented by graph autoencoders (GAEs), has increasingly garnered interest and demonstrated its potential in graph representation. However, existing generative SSL methods primarily concentrate on refining model designs, such as masking strategies and reconstruction objectives, while the latent representation generated by the encoder is scarcely explored beyond decoding by the decoder. Motivated by this observation, we identify and analyze the issues faced by previous works in utilizing latent representation and introduce GMAERank, a masked autoencoder with ranking learning. GMAERank presents a novel ranking strategy that leverages the latent representation. Specifically, GMAERank posits that the similarity between two augmented latent views should consistently outrank that between any augmented view and an unrelated random view-a condition typically achievable. Subsequently, GMAERank focus on the optimization of more difficult samples with the ranking strategy. In contrast to earlier research that involves auxiliary models for optimization, GMAERank is simple and lightweight. We conduct comprehensive experiments across node, edge, and graph-level tasks, and the results reveal that GMAERank achieves competitive performance and retains versatility when compared to state-of-the-art (SOTA) baselines.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Guo, Z., et al.: Linkless link prediction via relational distillation (2023)
Hasanzadeh, A., Hajiramezanali, E., Narayanan, K., Duffield, N., Zhou, M., Qian, X.: Semi-implicit graph variational auto-encoders. In: Advances in Neural Information Processing Systems, vol. 32 (2019)
Hassani, K., Khasahmadi, A.H.: Contrastive multi-view representation learning on graphs. In: International Conference on Machine Learning, pp. 4116–4126. PMLR (2020)
He, K., Chen, X., Xie, S., Li, Y., Dollár, P., Girshick, R.: Masked autoencoders are scalable vision learners. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 16000–16009 (2022)
Hou, Z., et al.: GraphMAE2: a decoding-enhanced masked self-supervised graph learner (2023)
Hou, Z., et al.: GraphMAE: self-supervised masked graph autoencoders. In: Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (2022)
Hu, Y., et al.: Do we really need contrastive learning for graph representation? (2023)
Hu, Y., Ouyang, S., Yang, Z., Liu, Y.: VIGraph: self-supervised learning for class-imbalanced node classification. arXiv preprint arXiv:2311.01191 (2023)
Hu, Y., Yang, Z., Ouyang, S., Liu, Y.: HGCVAE: integrating generative and contrastive learning for heterogeneous graph learning. arXiv preprint arXiv:2310.11102 (2023)
Hu, Z., Dong, Y., Wang, K., Chang, K.W., Sun, Y.: GPT-GNN: generative pre-training of graph neural networks. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 1857–1867 (2020)
Kingma, D.P., Welling, M.: Auto-encoding variational Bayes. arXiv preprint arXiv:1312.6114 (2013)
Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 (2016)
Kipf, T.N., Welling, M.: Variational graph auto-encoders. arXiv preprint arXiv:1611.07308 (2016)
Li, X., Ye, T., Shan, C., Li, D., Gao, M.: SeeGera: self-supervised semi-implicit graph variational auto-encoders with masking. In: Proceedings of the ACM Web Conference 2023, pp. 143–153 (2023)
Narayanan, A., Chandramohan, M., Venkatesan, R., Chen, L., Liu, Y., Jaiswal, S.: graph2vec: Learning distributed representations of graphs. arXiv preprint arXiv:1707.05005 (2017)
Oord, A.v.d., Li, Y., Vinyals, O.: Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748 (2018)
Pan, S., Hu, R., Long, G., Jiang, J., Yao, L., Zhang, C.: Adversarially regularized graph autoencoder for graph embedding (2019)
Qiu, J., et al.: GCC: graph contrastive coding for graph neural network pre-training. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 1150–1160 (2020)
Sun, F.Y., Hoffmann, J., Verma, V., Tang, J.: InfoGraph: unsupervised and semi-supervised graph-level representation learning via mutual information maximization. arXiv preprint arXiv:1908.01000 (2019)
Tan, Q., et al.: S2GAE: self-supervised graph autoencoders are generalizable learners with graph masking. In: Proceedings of the Sixteenth ACM International Conference on Web Search and Data Mining, pp. 787–795 (2023)
Thakoor, S., et al.: Large-scale representation learning on graphs via bootstrapping. arXiv preprint arXiv:2102.06514 (2021)
Velickovic, P., Cucurull, G., Casanova, A., Romero, A., Lio, P., Bengio, Y., et al.: Graph attention networks. Statistics 1050(20), 10–48550 (2017)
Veličković, P., Fedus, W., Hamilton, W.L., Liò, P., Bengio, Y., Hjelm, R.D.: Deep graph infomax. arXiv preprint arXiv:1809.10341 (2018)
Xu, D., Cheng, W., Luo, D., Chen, H., Zhang, X.: InfoGCL: information-aware graph contrastive learning. Adv. Neural. Inf. Process. Syst. 34, 30414–30425 (2021)
You, Y., Chen, T., Shen, Y., Wang, Z.: Graph contrastive learning automated. In: International Conference on Machine Learning, pp. 12121–12132. PMLR (2021)
You, Y., Chen, T., Sui, Y., Chen, T., Wang, Z., Shen, Y.: Graph contrastive learning with augmentations. Adv. Neural. Inf. Process. Syst. 33, 5812–5823 (2020)
Zhang, H., Wu, Q., Yan, J., Wipf, D., Yu, P.S.: From canonical correlation analysis to self-supervised graph neural networks. Adv. Neural. Inf. Process. Syst. 34, 76–89 (2021)
Zhu, Y., Xu, Y., Yu, F., Liu, Q., Wu, S., Wang, L.: Deep graph contrastive representation learning. arXiv preprint arXiv:2006.04131 (2020)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Hu, Y. et al. (2024). Advancing Latent Representation Ranking for Masked Graph Autoencoder. In: Onizuka, M., et al. Database Systems for Advanced Applications. DASFAA 2024. Lecture Notes in Computer Science, vol 14855. Springer, Singapore. https://doi.org/10.1007/978-981-97-5572-1_27
Download citation
DOI: https://doi.org/10.1007/978-981-97-5572-1_27
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-97-5571-4
Online ISBN: 978-981-97-5572-1
eBook Packages: Computer ScienceComputer Science (R0)