research-article

MetaLearning with Graph Neural Networks: Methods and Applications

Authors:

Debmalya Mandal,

Charu AggarwalAuthors Info & Claims

ACM SIGKDD Explorations Newsletter, Volume 23, Issue 2

Pages 13 - 22

https://doi.org/10.1145/3510374.3510379

Published: 03 January 2022 Publication History

Abstract

Graph Neural Networks (GNNs), a generalization of deep neural networks on graph data have been widely used in various domains, ranging from drug discovery to recommender systems. However, GNNs on such applications are limited when there are few available samples. Meta-learning has been an important framework to address the lack of samples in machine learning, and in recent years, researchers have started to apply meta-learning to GNNs. In this work, we provide a comprehensive survey of different metalearning approaches involving GNNs on various graph problems showing the power of using these two approaches together. We categorize the literature based on proposed architectures, shared representations, and applications. Finally, we discuss several exciting future research directions and open problems.

References

[1]

Jinheon Baek, Dong Bok Lee, and Sung Ju Hwang. "Learning to extrapolate knowledge: Transductive few-shot out-ofgraph link prediction". In: NeurIPS (2020).

[2]

Yunsheng Bai, Hao Ding, Song Bian, Ting Chen, Yizhou Sun, andWeiWang. "SimGNN: A Neural Network Approach to Fast Graph Similarity Computation". In: WSDM. 2019.

[3]

Yunsheng Bai, Derek Xu, Alex Wang, Ken Gu, Xueqing Wu, Agustin Marinovic, Christopher Ro, Yizhou Sun, and Wei Wang. "Fast detection of maximum common subgraph via deep q-learning". In: arXiv preprint arXiv:2002.03129 (2020).

[4]

Maria-Florina Balcan, Mikhail Khodak, and Ameet Talwalkar. "Provable guarantees for gradient-based meta-learning". In: ICML. 2019, pp. 424--433.

[5]

Jonathan Baxter. "A model of inductive bias learning". In: JAIR 12 (2000), pp. 149--198.

[6]

Karsten M Borgwardt, Cheng Soon Ong, Stefan Sch¨onauer, SVN Vishwanathan, Alex J Smola, and Hans-Peter Kriegel. "Protein function prediction via graph kernels". In: Bioinformatics 21 (2005), pp. i47--i56.

Digital Library

[7]

Avishek Joey Bose, Ankit Jain, Piero Molino, and William L Hamilton. "Meta-graph: Few shot link prediction via meta learning". In: arXiv preprint arXiv:1912.09867 (2019).

[8]

Davide Buffelli and Fabio Vandin. "A Meta-Learning Approach for Graph Representation Learning in Multi-Task Settings". In: arXiv preprint arXiv:2012.06755 (2020).

[9]

Shaosheng Cao, Wei Lu, and Qiongkai Xu. "Deep neural networks for learning graph representations". In: AAAI 30.1 (2016).

[10]

Quentin Cappart, Didier Ch´etelat, Elias Khalil, Andrea Lodi, Christopher Morris, and Petar Veli"ckovi´c. "Combinatorial optimization and reasoning with graph neural networks". In: arXiv preprint arXiv:2102.09544 (2021).

[11]

Jatin Chauhan, Deepak Nathani, and Manohar Kaul. "Few- Shot Learning on Graphs via Super-Classes based on Graph Spectral Measures". In: ICLR (2020).

[12]

Mingyang Chen, Wen Zhang, Wei Zhang, Qiang Chen, and Huajun Chen. "Meta Relational Learning for Few-Shot Link Prediction in Knowledge Graphs". In: EMNLP-IJCNLP. 2019, pp. 4208--4217.

[13]

Kurtland Chua, Qi Lei, and Jason D Lee. "How fine-tuning allows for effective meta-learning". In: arXiv:2105.02221 (2021).

[14]

Zhiyong Cui, Kristian Henrickson, Ruimin Ke, and Yinhai Wang. "Traffic graph convolutional recurrent neural network: A deep learning framework for network-scale traffic learning and forecasting". In: IEEE TITS (2019), pp. 4883--4894. SIGKDD Explorations Volume 23, Issue 2 20

[15]

Hanjun Dai, Elias B Khalil, Yuyu Zhang, Bistra Dilkina, and Le Song. "Learning combinatorial optimization algorithms over graphs". In: NeurIPS (2017).

[16]

Giulia Denevi, Carlo Ciliberto, Riccardo Grazzi, and Massimiliano Pontil. "Learning-to-learn stochastic gradient descent with biased regularization". In: ICML. 2019, pp. 1566-- 1575.

[17]

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. "Bert: Pre-training of deep bidirectional transformers for language understanding". In: NAACL-HLT (2019).

[18]

Kaize Ding, Jianling Wang, Jundong Li, James Caverlee, and Huan Liu. "Weakly-supervised Graph Meta-learning for Few-shot Node Classification". In: arXiv:2106.06873 (2021).

[19]

Kaize Ding, Jianling Wang, Jundong Li, Kai Shu, Chenghao Liu, and Huan Liu. "Graph prototypical networks for few-shot learning on attributed networks". In: CIKM. 2020, pp. 295--304.

Digital Library

[20]

Kaize Ding, Qinghai Zhou, Hanghang Tong, and Huan Liu. "Few-Shot Network Anomaly Detection via Cross-Network Meta-Learning". In: Proceedings of theWeb Conference 2021. 2021, 2448--2456.

[21]

Simon S Du, Wei Hu, Sham M Kakade, Jason D Lee, and Qi Lei. "Few-shot learning via learning the representation, provably". In: arXiv preprint arXiv:2002.09434 (2020).

[22]

Federico Errica, Marco Podda, Davide Bacciu, and Alessio Micheli. "A fair comparison of graph neural networks for graph classification". In: arXiv preprint arXiv:1912.09893 (2019).

[23]

Chelsea Finn, Pieter Abbeel, and Sergey Levine. "Modelagnostic meta-learning for fast adaptation of deep networks". In: ICML. 2017, pp. 1126--1135.

[24]

Chelsea Finn, Aravind Rajeswaran, Sham Kakade, and Sergey Levine. "Online meta-learning". In: ICML. 2019, pp. 1920-- 1930.

[25]

Vikas Garg, Stefanie Jegelka, and Tommi Jaakkola. "Generalization and representational limits of graph neural networks". In: ICML. PMLR. 2020, pp. 3419--3430.

[26]

Maxime Gasse, Didier Ch´etelat, Nicola Ferroni, Laurent Charlin, and Andrea Lodi. "Exact combinatorial optimization with graph convolutional neural networks". In: NeurIPS (2019).

[27]

Zhichun Guo, Chuxu Zhang, Wenhao Yu, John Herr, Olaf Wiest, Meng Jiang, and Nitesh V Chawla. "Few-Shot Graph Learning for Molecular Property Prediction". In: The Web Conference (2021).

[28]

Will Hamilton, Zhitao Ying, and Jure Leskovec. "Inductive representation learning on large graphs". In: NeurIPS. 2017, pp. 1024--1034.

[29]

William L Hamilton, Rex Ying, and Jure Leskovec. "Representation learning on graphs: Methods and applications". In: arXiv preprint arXiv:1709.05584 (2017).

[30]

Timothy Hospedales, Antreas Antoniou, Paul Micaelli, and Amos Storkey. "Meta-learning in neural networks: A survey". In: arXiv preprint arXiv:2004.05439 (2020).

[31]

Kexin Huang and Marinka Zitnik. "Graph meta learning via local subgraphs". In: NeurIPS (2020).

[32]

Dasol Hwang, Jinyoung Park, Sunyoung Kwon, Kyung-Min Kim, Jung-Woo Ha, and Hyunwoo J Kim. "Self-supervised Auxiliary Learning for Graph Neural Networks via Meta- Learning". In: arXiv preprint arXiv:2103.00771 (2021).

[33]

Shunyu Jiang, Fuli Feng, Weijian Chen, Xiang Li, and Xiangnan He. "Structure-Enhanced Meta-Learning For Few- Shot Graph Classification". In: arXiv preprint arXiv:2103.03547 (2021).

[34]

David Kempe, Jon Kleinberg, and ´ Eva Tardos. "Maximizing the spread of influence through a social network". In: KDD. 2003.

[35]

MKhodak,MBalcan, and A Talwalkar. "Adaptive Gradient- Based Meta-Learning Methods". In: Neural Information Processing Systems. 2019.

[36]

Thomas N Kipf and MaxWelling. "Semi-supervised classification with graph convolutional networks". In: ICLR (2017).

[37]

Thomas N Kipf and Max Welling. "Variational graph autoencoders". In: arXiv preprint arXiv:1611.07308 (2016).

[38]

Lin Lan, PinghuiWang, Xuefeng Du, Kaikai Song, Jing Tao, and Xiaohong Guan. "Node classification on graphs with few-shot novel labels via meta transformed network embedding". In: NeurIPS (2020).

[39]

Yujia Li, Daniel Tarlow, Marc Brockschmidt, and Richard Zemel. "Gated graph sequence neural networks". In: ICLR (2016).

[40]

Zhuwen Li, Qifeng Chen, and Vladlen Koltun. "Combinatorial optimization with graph convolutional networks and guided tree search". In: NeurIPS (2018).

[41]

Xiaodong Liu, Pengcheng He, Weizhu Chen, and Jianfeng Gao. "Multi-Task Deep Neural Networks for Natural Language Understanding". In: ACL. 2019.

[42]

Zemin Liu, Yuan Fang, Chenghao Liu, and Steven CH Hoi. "Relative and Absolute Location Embedding for Few-Shot Node Classification on Graph". In: AAAI (2021).

[43]

Zemin Liu,Wentao Zhang, Yuan Fang, Xinming Zhang, and Steven CH Hoi. "Towards locality-aware meta-learning of tail node embeddings on networks". In: CIKM. 2020, pp. 975-- 984.

Digital Library

[44]

Ning Ma, Jiajun Bu, Jieyu Yang, Zhen Zhang, Chengwei Yao, Zhi Yu, Sheng Zhou, and Xifeng Yan. "Adaptive-Step Graph Meta-Learner for Few-Shot Graph Classification". In: Proceedings of the 29th ACM International Conference on Information & Knowledge Management. 2020, 1055--1064.

[45]

Sahil Manchanda, Akash Mittal, Anuj Dhawan, Sourav Medya, Sayan Ranu, and Ambuj Singh. "GCOMB: Learning Budget-constrained Combinatorial Algorithms over Billionsized Graphs". In: NeurIPS (2020).

[46]

Andreas Maurer, Massimiliano Pontil, and Bernardino Romera- Paredes. "The benefit of multitask representation learning". In: Journal of Machine Learning Research 17.81 (2016), pp. 1--32.

Digital Library

[47]

Sourav Medya, Jithin Vachery, Sayan Ranu, and Ambuj Singh. "Noticeable network delay minimization via node upgrades". In: VLDB (2018).

[48]

Anusha Nagabandi, Kurt Konolige, Sergey Levine, and Vikash Kumar. "Deep dynamics models for learning dexterous manipulation". In: CoRL. 2020.

[49]

Sunil Nishad, Shubhangi Agarwal, Arnab Bhattacharya, and Sayan Ranu. "GraphReach: Locality-Aware Graph Neural Networks using Reachability Estimations". In: IJCAI. 2021. SIGKDD Explorations Volume 23, Issue 2 21

[50]

Zheyi Pan, Wentao Zhang, Yuxuan Liang, Weinan Zhang, Yong Yu, Junbo Zhang, and Yu Zheng. "Spatio-Temporal Meta Learning for Urban Traffic Prediction". In: TKDE (2020).

[51]

Massimiliano Pontil and Andreas Maurer. "Excess risk bounds for multitask learning with trace norm regularization". In: COLT. 2013, pp. 55--76.

[52]

Sachin Ravi and Hugo Larochelle. "Optimization as a model for few-shot learning". In: ICLR. 2017.

[53]

Franco Scarselli, Marco Gori, Ah Chung Tsoi, Markus Hagenbuchner, and Gabriele Monfardini. "The graph neural network model". In: IEEE transactions on neural networks 20.1 (2008), pp. 61--80.

[54]

Franco Scarselli, Ah Chung Tsoi, and Markus Hagenbuchner. "The Vapnik--Chervonenkis dimension of graph and recursive neural networks". In: Neural Networks 108 (2018), pp. 248--259.

[55]

J¨urgen Schmidhuber. "Evolutionary principles in self-referential learning, or on learning how to learn: the meta-meta-... hook". PhD thesis. Technische Universit¨at M¨unchen, 1987.

[56]

Oleksandr Shchur, Maximilian Mumme, Aleksandar Bojchevski, and Stephan G¨unnemann. "Pitfalls of graph neural network evaluation". In: arXiv preprint arXiv:1811.05868 (2018).

[57]

Jonathan M Stokes, Kevin Yang, Kyle Swanson, Wengong Jin, Andres Cubillos-Ruiz, et al. "A deep learning approach to antibiotic discovery". In: Cell (2020), pp. 688--702.

[58]

Jie Tang, Jing Zhang, Limin Yao, Juanzi Li, Li Zhang, and Zhong Su. "Arnetminer: extraction and mining of academic social networks". In: KDD. 2008.

Digital Library

[59]

Nilesh Tripuraneni, Chi Jin, and Michael I Jordan. "Provable meta-learning of linear representations". In: arXiv preprint arXiv:2002.11684 (2020).

[60]

Nilesh Tripuraneni, Michael Jordan, and Chi Jin. "On the Theory of Transfer Learning: The Importance of Task Diversity". In: NeurIPS 33 (2020).

[61]

Petar Veli?ckovi´c, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. "Graph attention networks". In: ICLR (2018).

[62]

Ning Wang, Minnan Luo, Kaize Ding, Lingling Zhang, Jundong Li, and Qinghua Zheng. "Graph Few-shot Learning with Attribute Matching". In: CIKM. 2020, pp. 1545--1554.

Digital Library

[63]

Yan Wang, Wei-Lun Chao, Kilian Q Weinberger, and Laurens van der Maaten. "Simpleshot: Revisiting nearest-neighbor classification for few-shot learning". In: arXiv:1911.04623 (2019).

[64]

Zhihao Wen, Yuan Fang, and Zemin Liu. "Meta-Inductive Node Classification across Graphs". In: arXiv:2105.06725 (2021).

[65]

Bryan Wilder, Han Ching Ou, Kayla de la Haye, and Milind Tambe. "Optimizing Network Structure for Preventative Health". In: AAMAS. 2018.

[66]

NanWu, Jason Phang, Jungkyu Park, Yiqiu Shen, et al. "Deep neural networks improve radiologists' performance in breast cancer screening". In: IEEE transactions on medical imaging 39.4 (2019), pp. 1184--1194.

[67]

Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long, Chengqi Zhang, and S Yu Philip. "A comprehensive survey on graph neural networks". In: IEEE transactions on neural networks and learning systems (2020).

[68]

Keyulu Xu,Weihua Hu, Jure Leskovec, and Stefanie Jegelka. "How powerful are graph neural networks?" In: ICLR (2018).

[69]

Pinar Yanardag and SVN Vishwanathan. "Deep graph kernels". In: Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining. 2015, pp. 1365--1374.

Digital Library

[70]

Huaxiu Yao, Chuxu Zhang, Ying Wei, Meng Jiang, Suhang Wang, Junzhou Huang, Nitesh Chawla, and Zhenhui Li. "Graph few-shot learning via knowledge transfer". In: AAAI. 2020, pp. 6656--6663.

[71]

Jiaxuan You, Rex Ying, and Jure Leskovec. "Position-aware Graph Neural Networks". In: ICML. 2019, pp. 7134--7143.

[72]

Jiani Zhang, Xingjian Shi, Junyuan Xie, Hao Ma, Irwin King, and Dit-Yan Yeung. "Gaan: Gated attention networks for learning on large and spatiotemporal graphs". In: UAI (2018).

[73]

Muhan Zhang and Yixin Chen. "Link prediction based on graph neural networks". In: NeurIPS (2018).

[74]

Xi Sheryl Zhang, Fengyi Tang, Hiroko H Dodge, Jiayu Zhou, and Fei Wang. "Metapred: Meta-learning for clinical risk prediction with limited patient electronic health records". In: KDD. 2019.

[75]

Fan Zhou, Chengtai Cao, Goce Trajcevski, Kunpeng Zhang, Ting Zhong, and Ji Geng. "Fast network alignment via graph meta-learning". In: INFOCOM. 2020, pp. 686--695.

[76]

Fan Zhou, Chengtai Cao, Kunpeng Zhang, Goce Trajcevski, Ting Zhong, and Ji Geng. "Meta-Gnn: On Few-Shot Node Classification in Graph Meta-Learning". In: CIKM. 2019, pp. 2357--2360.

[77]

Jie Zhou, Ganqu Cui, Zhengyan Zhang, Cheng Yang, Zhiyuan Liu, LifengWang, Changcheng Li, and Maosong Sun. "Graph neural networks: A review of methods and applications". In: arXiv preprint arXiv:1812.08434 (2018).

Cited By

Ma HFan BNg BLam C(2024)VL-Meta: Vision-Language Models for Multimodal Meta-LearningMathematics10.3390/math1202028612:2(286)Online publication date: 16-Jan-2024
https://doi.org/10.3390/math12020286
Dai YSun MWang X(2024)Pre-training Graph Neural Networks via Weighted Meta Learning2024 International Joint Conference on Neural Networks (IJCNN)10.1109/IJCNN60899.2024.10650709(1-8)Online publication date: 30-Jun-2024
https://doi.org/10.1109/IJCNN60899.2024.10650709
Chen ZWang YMa FYuan HWang X(2024)GPL-GNNKnowledge-Based Systems10.1016/j.knosys.2024.111391286:COnline publication date: 17-Apr-2024
https://dl.acm.org/doi/10.1016/j.knosys.2024.111391
Show More Cited By

MetaLearning with Graph Neural Networks: Methods and Applications
1. Computing methodologies
  1. Machine learning
    1. Machine learning approaches

Recommendations

Graph Neural Networks: Taxonomy, Advances, and Trends
Graph neural networks provide a powerful toolkit for embedding real-world graphs into low-dimensional spaces according to specific tasks. Up to now, there have been several surveys on this topic. However, they usually lay emphasis on different angles so ...
Granular neural networks

Fuzzy neural networks (FNNs) and rough neural networks (RNNs) both have been hot research topics in the artificial intelligence in recent years. The former imitates the human brain in dealing with problems, the other takes advantage of rough set theory ...
Toward the analysis of graph neural networks
ICSE-NIER '22: Proceedings of the ACM/IEEE 44th International Conference on Software Engineering: New Ideas and Emerging Results

Graph Neural Networks (GNNs) have recently emerged as an effective framework for representing and analyzing graph-structured data. GNNs have been applied to many real-world problems such as knowledge graph analysis, social networks recommendation, and ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM SIGKDD Explorations Newsletter

ACM SIGKDD Explorations Newsletter Volume 23, Issue 2

December 2021

22 pages

ISSN:1931-0145

EISSN:1931-0153

DOI:10.1145/3510374

Editors:
Hanghang Tong
Arizona State University
,
Xin Luna Dong
Google
,
Ankur Teredesai
University of Washington Tacoma
,
Reza Zafarani
Syracuse University

Issue’s Table of Contents

Copyright © 2022 Copyright is held by the owner/author(s).

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 03 January 2022

Published in SIGKDD Volume 23, Issue 2

Check for updates

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

10
Total Citations
View Citations
884
Total Downloads

Downloads (Last 12 months)187
Downloads (Last 6 weeks)11

Reflects downloads up to 21 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Ma HFan BNg BLam C(2024)VL-Meta: Vision-Language Models for Multimodal Meta-LearningMathematics10.3390/math1202028612:2(286)Online publication date: 16-Jan-2024
https://doi.org/10.3390/math12020286
Dai YSun MWang X(2024)Pre-training Graph Neural Networks via Weighted Meta Learning2024 International Joint Conference on Neural Networks (IJCNN)10.1109/IJCNN60899.2024.10650709(1-8)Online publication date: 30-Jun-2024
https://doi.org/10.1109/IJCNN60899.2024.10650709
Chen ZWang YMa FYuan HWang X(2024)GPL-GNNKnowledge-Based Systems10.1016/j.knosys.2024.111391286:COnline publication date: 17-Apr-2024
https://dl.acm.org/doi/10.1016/j.knosys.2024.111391
Long MChen SWang J(2024)Locally-adaptive mapping for network alignment via meta-learningInformation Processing & Management10.1016/j.ipm.2024.10381761:5(103817)Online publication date: Sep-2024
https://doi.org/10.1016/j.ipm.2024.103817
Madadi BHomem de Almeida Correia G(2024)A hybrid deep-learning-metaheuristic framework for bi-level network design problemsExpert Systems with Applications: An International Journal10.1016/j.eswa.2023.122814243:COnline publication date: 25-Jun-2024
https://dl.acm.org/doi/10.1016/j.eswa.2023.122814
Jin BLi SHuang J(2024)GraphSAGE-Based Spammer Detection Using Social Attribute RelationshipTechnologies and Applications of Artificial Intelligence10.1007/978-981-97-1711-8_23(300-313)Online publication date: 28-Mar-2024
https://doi.org/10.1007/978-981-97-1711-8_23
Dou BZhu ZMerkurjev EKe LChen LJiang JZhu YLiu JZhang BWei G(2023)Machine Learning Methods for Small Data Challenges in Molecular ScienceChemical Reviews10.1021/acs.chemrev.3c00189123:13(8736-8780)Online publication date: 29-Jun-2023
https://doi.org/10.1021/acs.chemrev.3c00189
Das BT‏S‏B‏ S(2023)Multi-contextual learning in disinformation research: A review of challenges, approaches, and opportunitiesOnline Social Networks and Media10.1016/j.osnem.2023.10024734-35(100247)Online publication date: May-2023
https://doi.org/10.1016/j.osnem.2023.100247
Sarkar DRoy SMalakar SSarkar R(2023)A modified GNN architecture with enhanced aggregator and Message Passing FunctionsEngineering Applications of Artificial Intelligence10.1016/j.engappai.2023.106077122:COnline publication date: 1-Jun-2023
https://dl.acm.org/doi/10.1016/j.engappai.2023.106077
Yu ZLv J(2023)A PID controller method for meta-learning about aerospace target classificationMultimedia Tools and Applications10.1007/s11042-023-17022-083:13(39217-39233)Online publication date: 7-Oct-2023
https://doi.org/10.1007/s11042-023-17022-0

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents