research-article

Graph Few-shot Learning with Attribute Matching

Authors:

Lingling Zhang,

Qinghua ZhengAuthors Info & Claims

CIKM '20: Proceedings of the 29th ACM International Conference on Information & Knowledge Management

Pages 1545 - 1554

https://doi.org/10.1145/3340531.3411923

Published: 19 October 2020 Publication History

Abstract

Due to the expensive cost of data annotation, few-shot learning has attracted increasing research interests in recent years. Various meta-learning approaches have been proposed to tackle this problem and have become the de facto practice. However, most of the existing approaches along this line mainly focus on image and text data in the Euclidean domain. However, in many real-world scenarios, a vast amount of data can be represented as attributed networks defined in the non-Euclidean domain, and the few-shot learning studies in such structured data have largely remained nascent. Although some recent studies have tried to combine meta-learning with graph neural networks to enable few-shot learning on attributed networks, they fail to account for the unique properties of attributed networks when creating diverse tasks in the meta-training phase---the feature distributions of different tasks could be quite different as instances (i.e., nodes) do not follow the data i.i.d. assumption on attributed networks. Hence, it may inevitably result in suboptimal performance in the meta-testing phase. To tackle the aforementioned problem, we propose a novel graph meta-learning framework--Attribute Matching Meta-learning Graph Neural Networks (AMM-GNN). Specifically, the proposed AMM-GNN leverages an attribute-level attention mechanism to capture the distinct information of each task and thus learns more effective transferable knowledge for meta-learning. We conduct extensive experiments on real-world datasets under a wide range of settings and the experimental results demonstrate the effectiveness of the proposed AMM-GNN framework.

Supplementary Material

MP4 File (3340531.3411923.mp4)

The presentation video of paper "Graph Few-shot Learning with Attribute Matching".

Download
38.27 MB

References

[1]

Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473 (2014).

[2]

Indrajit Bhattacharya and Lise Getoor. 2007. Collective entity resolution in relational data. ACM Transactions on Knowledge Discovery from Data (TKDD), Vol. 1, 1 (2007), 5--es.

[3]

Avishek Joey Bose, Ankit Jain, Piero Molino, and William L Hamilton. 2019. Meta-graph: Few shot link prediction via meta learning. arXiv preprint arXiv:1912.09867 (2019).

[4]

Joan Bruna, Wojciech Zaremba, Arthur Szlam, and Yann LeCun. 2014. Spectral networks and locally connected networks on graphs. In International Conference on Learning Representations (ICLR).

[5]

Qi Cai, Yingwei Pan, Ting Yao, Chenggang Yan, and Tao Mei. 2018. Memory matching networks for one-shot image recognition. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[6]

Shaosheng Cao, Wei Lu, and Qiongkai Xu. 2015. GraRep: Learning graph representations with global structural information. In ACM International on Conference on Information and Knowledge Management (CIKM).

Digital Library

[7]

Jie Chen, Tengfei Ma, and Cao Xiao. 2018. Fastgcn: fast learning with graph convolutional networks via importance sampling. arXiv preprint arXiv:1801.10247 (2018).

[8]

Noy Cohen-Shapira, Lior Rokach, Bracha Shapira, Gilad Katz, and Roman Vainshtein. 2019. AutoGRD: Model recommendation through graphical dataset representation. In ACM International Conference on Information and Knowledge Management (CIKM).

Digital Library

[9]

Kaize Ding, Jundong Li, Nitin Agarwal, and Huan Liu. 2020 a. Inductive Anomaly Detection on Attributed Networks. In Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI. 1288--1294.

[10]

Kaize Ding, Jundong Li, Rohit Bhanushali, and Huan Liu. 2019. Deep anomaly detection on attributed networks. In Proceedings of the 2019 SIAM International Conference on Data Mining (SDM). 594--602.

[11]

Kaize Ding, Jianling Wang, Jundong Li, Kai Shu, Chenghao Liu, and Huan Liu. 2020 b. Graph Prototypical Networks for Few-shot Learning on Attributed Networks. arXiv preprint arXiv:2006.12739 (2020).

[12]

Li Fei-Fei, Rob Fergus, and Pietro Perona. 2006. One-shot learning of object categories. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), Vol. 28, 4 (2006), 594--611.

Digital Library

[13]

Michael Fink. 2005. Object classification from a single example utilizing class relevance metrics. In Conference on Neural Information Processing Systems (NIPS).

[14]

Chelsea Finn, Pieter Abbeel, and Sergey Levine. 2017. Model-agnostic meta-learning for fast adaptation of deep networks. In International Conference on Machine Learning (ICML).

[15]

Madhavi K Ganapathiraju, Mohamed Thahir, Adam Handen, Saumendra N Sarkar, Robert A Sweet, Vishwajit L Nimgaonkar, Christine E Loscher, Eileen M Bauer, and Srilakshmi Chaparala. 2016. Schizophrenia interactome with 504 novel protein--protein interactions. NPJ schizophrenia, Vol. 2, 1 (2016), 1--10.

[16]

Hongchang Gao and Heng Huang. 2018. Deep Attributed Network Embedding. In International Joint Conference on Artificial Intelligence (IJCAI). 3364--3370.

[17]

Hongyang Gao, Zhengyang Wang, and Shuiwang Ji. 2018. Large-scale learnable graph convolutional networks. In ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (SIGKDD). 1416--1424.

Digital Library

[18]

Victor Garcia and Joan Bruna. 2017. Few-shot learning with graph neural networks. arXiv preprint arXiv:1711.04043 (2017).

[19]

Marco Gori, Gabriele Monfardini, and Franco Scarselli. 2005. A new model for learning in graph domains. In International Joint Conference on Neural Networks (IJCNN), Vol. 2. IEEE, 729--734.

[20]

Aditya Grover and Jure Leskovec. 2016. node2vec: Scalable feature learning for networks. In ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (SIGKDD).

Digital Library

[21]

Will Hamilton, Zhitao Ying, and Jure Leskovec. 2017. Inductive representation learning on large graphs. In Conference on Neural Information Processing Systems (NIPS).

[22]

Douglas M Hawkins. 2004. The problem of overfitting. Journal of Chemical Information and Computer Sciences, Vol. 44, 1 (2004), 1--12.

[23]

Xiao Huang, Jundong Li, and Xia Hu. 2017a. Accelerated attributed network embedding. In Proceedings of the 2017 SIAM International Conference on Data Mining (SDM). 633--641.

[24]

Xiao Huang, Jundong Li, and Xia Hu. 2017b. Label informed attributed network embedding. In ACM International Conference on Web Search and Data Mining. 731--739 (WSDM).

Digital Library

[25]

Vidur Joshi, Matthew Peters, and Mark Hopkins. 2018. Extending a parser to distant domains using a few dozen partially annotated examples. arXiv preprint arXiv:1805.06556 (2018).

[26]

Łukasz Kaiser, Ofir Nachum, Aurko Roy, and Samy Bengio. 2017. Learning to remember rare events. arXiv preprint arXiv:1703.03129 (2017).

[27]

Thomas N Kipf and Max Welling. 2017. Semi-supervised classification with graph convolutional networks. In International Conference on Learning Representations (ICLR).

[28]

Gregory Koch, Richard Zemel, and Ruslan Salakhutdinov. 2015. Siamese neural networks for one-shot image recognition. In ICML deep learning workshop, Vol. 2. Lille.

[29]

Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. 2012. Imagenet classification with deep convolutional neural networks. In Conference on Neural Information Processing Systems (NIPS).

[30]

Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. 1998. Gradient-based learning applied to document recognition. Proc. IEEE, Vol. 86, 11 (1998), 2278--2324.

[31]

Jundong Li, Harsh Dani, Xia Hu, Jiliang Tang, Yi Chang, and Huan Liu. 2017. Attributed network embedding for learning in a dynamic environment. In Proceedings of the 2017 ACM on Conference on Information and Knowledge Management (CIKM). 387--396.

Digital Library

[32]

Ruoyu Li, Sheng Wang, Feiyun Zhu, and Junzhou Huang. 2018. Adaptive graph convolutional neural networks. In AAAI Conference on Artificial Intelligence (AAAI).

[33]

Julian McAuley, Rahul Pandey, and Jure Leskovec. 2015. Inferring networks of substitutable and complementary products. In ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (SIGKDD).

Digital Library

[34]

Jeffrey Pennington, Richard Socher, and Christopher D Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). 1532--1543.

[35]

Bryan Perozzi, Rami Al-Rfou, and Steven Skiena. 2014. Deepwalk: Online learning of social representations. In ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (SIGKDD).

Digital Library

[36]

Joseph J Pfeiffer III, Sebastian Moreno, Timothy La Fond, Jennifer Neville, and Brian Gallagher. 2014. Attributed graph models: Modeling network structure with correlated attributes. In International Conference on World Wide Web (WWW). 831--842.

[37]

Sachin Ravi and Hugo Larochelle. 2017. Optimization as a model for few-shot learning. In International Conference on Learning Representations (ICLR).

[38]

Anthony Rios and Ramakanth Kavuluru. 2018. Few-shot and zero-shot multi-label learning for structured label spaces. In Conference on Empirical Methods in Natural Language Processing (EMNLP). 3132--3142.

[39]

Franco Scarselli, Marco Gori, Ah Chung Tsoi, Markus Hagenbuchner, and Gabriele Monfardini. 2008. The graph neural network model. IEEE Transactions on Neural Networks, Vol. 20, 1 (2008), 61--80.

Digital Library

[40]

Prithviraj Sen, Galileo Namata, Mustafa Bilgic, Lise Getoor, Brian Galligher, and Tina Eliassi-Rad. 2008. Collective classification in network data. AI Magazine, Vol. 29, 3 (2008), 93--93.

Digital Library

[41]

Jake Snell, Kevin Swersky, and Richard Zemel. 2017. Prototypical networks for few-shot learning. In Conference on Neural Information Processing Systems (NIPS).

[42]

Flood Sung, Yongxin Yang, Li Zhang, Tao Xiang, Philip HS Torr, and Timothy M Hospedales. 2018. Learning to compare: Relation network for few-shot learning. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[43]

Jie Tang, Jing Zhang, Limin Yao, Juanzi Li, Li Zhang, and Zhong Su. 2008. Arnetminer: extraction and mining of academic social networks. In ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (SIGKDD).

Digital Library

[44]

Igor V Tetko, David J Livingstone, and Alexander I Luik. 1995. Neural network studies. 1. Comparison of overfitting and overtraining. Journal of Chemical Information and Computer Sciences (JCIM), Vol. 35, 5 (1995), 826--833.

[45]

Petar Velivc ković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. 2017. Graph attention networks. arXiv preprint arXiv:1710.10903 (2017).

[46]

Oriol Vinyals, Charles Blundell, Timothy Lillicrap, Daan Wierstra, et al. 2016. Matching networks for one shot learning. In Conference on Neural Information Processing Systems (NIPS).

[47]

Chun Wang, Shirui Pan, Guodong Long, Xingquan Zhu, and Jing Jiang. 2017. MGAE: Marginalized graph autoencoder for graph clustering. In ACM International Conference on Information and Knowledge Management (CIKM).

Digital Library

[48]

Fei Wang and Changshui Zhang. 2007. Label propagation through linear neighborhoods. IEEE Transactions on Knowledge and Data Engineering, Vol. 20, 1 (2007), 55--67.

Digital Library

[49]

Jianling Wang, Kaize Ding, Liangjie Hong, Huan Liu, and James Caverlee. 2020 a. Next-item Recommendation with Sequential Hypergraphs. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR). 1101--1110.

Digital Library

[50]

Yaqing Wang, Quanming Yao, James T Kwok, and Lionel M Ni. 2020 b. Generalizing from a few examples: A survey on few-shot learning. ACM Computing Surveys (CSUR), Vol. 53, 3 (2020), 1--34.

Digital Library

[51]

Felix Wu, Tianyi Zhang, Amauri Holanda de Souza Jr, Christopher Fifty, Tao Yu, and Kilian Q Weinberger. 2019. Simplifying graph convolutional networks. In International Conference on Machine Learning (ICML).

[52]

Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long, Chengqi Zhang, and S Yu Philip. 2020. A comprehensive survey on graph neural networks. IEEE Transactions on Neural Networks and Learning Systems (TNNLS) (2020).

[53]

Zhilin Yang, William Cohen, and Ruslan Salakhudinov. 2016. Revisiting semi-supervised learning with graph embeddings. In International Conference on Machine Learning (ICML). 40--48.

[54]

Huaxiu Yao, Chuxu Zhang, Ying Wei, Meng Jiang, Suhang Wang, Junzhou Huang, Nitesh V Chawla, and Zhenhui Li. 2020. Graph few-shot learning via knowledge transfer. In AAAI Conference on Artificial Intelligence (AAAI).

[55]

Lingling Zhang, Xiaojun Chang, Jun Liu, Minnan Luo, and Alexander Hauptmann. 2020 a. ZSTAD: Zero-shot temporal activity detection. (2020).

[56]

Lingling Zhang, Jun Liu, Minnan Luo, Xiaojun Chang, and Qinghua Zheng. 2018. Deep semisupervised zero-shot learning with maximum mean discrepancy. Neural Computation, Vol. 30, 5 (2018), 1426--1447.

Digital Library

[57]

Si Zhang and Hanghang Tong. 2016. Final: Fast attributed network alignment. In ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (SIGKDD). 1345--1354.

Digital Library

[58]

Ziwei Zhang, Peng Cui, and Wenwu Zhu. 2020 b. Deep learning on graphs: A survey. IEEE Transactions on Knowledge and Data Engineering (2020).

[59]

Fan Zhou, Chengtai Cao, Kunpeng Zhang, Goce Trajcevski, Ting Zhong, and Ji Geng. 2019. Meta-GNN: On few-shot node classification in graph meta-learning. In ACM International Conference on Information and Knowledge Management (CIKM).

Digital Library

[60]

Jie Zhou, Ganqu Cui, Zhengyan Zhang, Cheng Yang, Zhiyuan Liu, Lifeng Wang, Changcheng Li, and Maosong Sun. 2018. Graph neural networks: A review of methods and applications. arXiv preprint arXiv:1812.08434 (2018).

[61]

Linchao Zhu and Yi Yang. 2018. Compound memory networks for few-shot video classification. In European Conference on Computer Vision (ECCV).

Digital Library

[62]

Xiaojin Zhu. 2005. Semi-supervised learning with graphs. (2005).

Cited By

Wang JLin TWu HWang P(2024)AGProto: Adaptive Graph ProtoNet towards Sample Adaption for Few-Shot Malware ClassificationElectronics10.3390/electronics1305093513:5(935)Online publication date: 29-Feb-2024
https://doi.org/10.3390/electronics13050935
Zhang KShin DSeo DCai Z(2024)Few-Shot Graph Classification with Structural-Enhanced Contrastive Learning for Graph Data Copyright ProtectionTsinghua Science and Technology10.26599/TST.2023.901007129:2(605-616)Online publication date: Apr-2024
https://doi.org/10.26599/TST.2023.9010071
Fang YZhao XXiao Wde Rijke M(2024)Few-shot Learning for Heterogeneous Information NetworksACM Transactions on Information Systems10.1145/364931142:4(1-24)Online publication date: 27-Feb-2024
https://dl.acm.org/doi/10.1145/3649311
Show More Cited By

Index Terms

Graph Few-shot Learning with Attribute Matching
1. Computing methodologies
  1. Machine learning
    1. Learning paradigms
      1. Multi-task learning
        Transfer learning
    2. Machine learning approaches
      1. Neural networks

Recommendations

Meta-GNN: On Few-shot Node Classification in Graph Meta-learning
CIKM '19: Proceedings of the 28th ACM International Conference on Information and Knowledge Management

Meta-learning has received a tremendous recent attention as a possible approach for mimicking human intelligence, i.e., acquiring new knowledge and skills with little or even no demonstration. Most of the existing meta-learning methods are proposed to ...
Task-Equivariant Graph Few-shot Learning
KDD '23: Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

Although Graph Neural Networks (GNNs) have been successful in node classification tasks, their performance heavily relies on the availability of a sufficient number of labeled nodes per class. In real-world situations, not all classes have many labeled ...
Robust Graph Meta-Learning for Weakly Supervised Few-Shot Node Classification
Graph machine learning (Graph ML) models typically require abundant labeled instances to provide sufficient supervision signals, which is commonly infeasible in real-world scenarios since labeled data for newly emerged concepts (e.g., new categorizations ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

CIKM '20: Proceedings of the 29th ACM International Conference on Information & Knowledge Management

October 2020

3619 pages

ISBN:9781450368599

DOI:10.1145/3340531

General Chairs:
Mathieu d'Aquin
DSI, Insight, NUI Galway, Ireland
,
Stefan Dietze
GESIS, Cologne, Germany, Heinrich-Heine-University Düsseldorf, Germany, L3S Research Center, Germany
,
Program Chairs:
Claudia Hauff
TU Delft, The Netherlands
,
Edward Curry
DSI, Insight, NUI Galway, Ireland
,
Philippe Cudre Mauroux
eXascale, University of Fribourg, Switzerland

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 19 October 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Project of China Knowledge Center for Engineering Science and Technology
Innovation Research Team of Ministry of Education
National Nature Science Foundation of China
Innovative Research Group of the National Natural Science Foundation of China

Conference

CIKM '20

Sponsor:

CIKM '20: The 29th ACM International Conference on Information and Knowledge Management

October 19 - 23, 2020

Virtual Event, Ireland

Acceptance Rates

Overall Acceptance Rate 1,861 of 8,427 submissions, 22%

Upcoming Conference

CIKM '25

Sponsor:
sigir
sigir

The 34th ACM International Conference on Information and Knowledge Management

November 10 - 14, 2025

Seoul , Republic of Korea

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

35
Total Citations
View Citations
930
Total Downloads

Downloads (Last 12 months)132
Downloads (Last 6 weeks)19

Reflects downloads up to 12 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Wang JLin TWu HWang P(2024)AGProto: Adaptive Graph ProtoNet towards Sample Adaption for Few-Shot Malware ClassificationElectronics10.3390/electronics1305093513:5(935)Online publication date: 29-Feb-2024
https://doi.org/10.3390/electronics13050935
Zhang KShin DSeo DCai Z(2024)Few-Shot Graph Classification with Structural-Enhanced Contrastive Learning for Graph Data Copyright ProtectionTsinghua Science and Technology10.26599/TST.2023.901007129:2(605-616)Online publication date: Apr-2024
https://doi.org/10.26599/TST.2023.9010071
Fang YZhao XXiao Wde Rijke M(2024)Few-shot Learning for Heterogeneous Information NetworksACM Transactions on Information Systems10.1145/364931142:4(1-24)Online publication date: 27-Feb-2024
https://dl.acm.org/doi/10.1145/3649311
Ding KWang JLi JCaverlee JLiu H(2024)Robust Graph Meta-Learning for Weakly Supervised Few-Shot Node ClassificationACM Transactions on Knowledge Discovery from Data10.1145/363026018:4(1-18)Online publication date: 13-Feb-2024
https://dl.acm.org/doi/10.1145/3630260
Yu XZhou CFang YZhang XChua TNgo CKa-Wei Lee RKumar RLauw H(2024)MultiGPrompt for Multi-Task Pre-Training and Prompting on GraphsProceedings of the ACM Web Conference 202410.1145/3589334.3645423(515-526)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3645423
Zhao FHuang TWang D(2024)Graph Few-Shot Learning via Restructuring Task GraphIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2022.317884935:1(1415-1422)Online publication date: Jan-2024
https://doi.org/10.1109/TNNLS.2022.3178849
Yu XLiu ZFang YLiu ZChen SZhang X(2024)Generalized Graph Prompt: Toward a Unification of Pre-Training and Downstream Tasks on GraphsIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2024.341910936:11(6237-6250)Online publication date: Nov-2024
https://doi.org/10.1109/TKDE.2024.3419109
Wang YHuang CLi MHuang QWu XWu J(2024)AG-MetaPattern Recognition10.1016/j.patcog.2024.110387151:COnline publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1016/j.patcog.2024.110387
Ding KLiu YZhang CWang J(2024)Data‐efficient graph learning: Problems, progress, and prospectsAI Magazine10.1002/aaai.12200Online publication date: 18-Oct-2024
https://doi.org/10.1002/aaai.12200
Xu XDu JSong JXue Z(2023)InfoMax Classification-Enhanced Learnable Network for Few-Shot Node ClassificationElectronics10.3390/electronics1201023912:1(239)Online publication date: 3-Jan-2023
https://doi.org/10.3390/electronics12010239
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents