Curriculum Meta Learning: Learning to Learn from Easy to Hard
Authors: 
Jiangbo Liu, Zhenyong FuAuthors Info & Claims
Pages 1571 - 1576
Abstract
Meta-learning is a machine learning paradigm that extracts crosstask knowledge by learning a large number of subtasks, to fast adapt to new tasks. Many meta-learning methods are widely applied in few-shot classification. These methods adopt an episodic training strategy, and the learning subtasks are sampled uniformly from the task distribution. In this paper, we explore the effect of the order of training subtasks on the performance of different meta-learning algorithms and propose a curriculum learning framework to improve the generalization performance. We define the hardness of subtasks at the class level and guide the model to learn training subtasks from easy to hard. We evaluate our curriculum learning framework on two few-shot classification benchmarks (mini-ImageNet and FC100), and it achieves improvements across different meta-learning algorithms and datasets. In the cross-domain scenario, we compare the performance of different meta learning algorithms under three curriculum settings. The results show that our CL approach improves significantly the generalization performance of different meta-learning methods.
References
[1]
He K, Zhang X, Ren S, et al. Deep residual learning for image recognition[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2016: 770--778.
[2]
Ren S, He K, Girshick R, et al. Faster r-cnn: Towards real-time object detection with region proposal networks[J]. Advances in neural information processing systems, 2015, 28: 91--99.
[3]
Deng J, Dong W, Socher R, et al. Imagenet: A large-scale hierarchical image database[C]//2009 IEEE conference on computer vision and pattern recognition. Ieee, 2009: 248--255.
[4]
Ravi S, Larochelle H. Optimization as a model for few-shot learning[J]. 2016.
[5]
Vanschoren J. Meta-learning: A survey[J]. arXiv preprint arXiv:1810.03548, 2018.
[6]
Fei-Fei L, Fergus R, Perona P. One-shot learning of object categories[J]. IEEE transactions on pattern analysis and machine intelligence, 2006, 28(4): 594--611.
[7]
Smith L B, Slone L K. A developmental approach to machine learning?[J]. Frontiers in psychology, 2017, 8: 2124.
[8]
Finn C, Abbeel P, Levine S. Model-agnostic meta-learning for fast adaptation of deep networks[C]//International Conference on Machine Learning. PMLR, 2017: 1126--1135.
[9]
Snell J, Swersky K, Zemel R S. Prototypical networks for few-shot learning[J]. arXiv preprint arXiv:1703.05175, 2017.
[10]
Sung F, Yang Y, Zhang L, et al. Learning to compare: Relation network for fewshot learning[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2018: 1199--1208.
[11]
Ye H J, Hu H, Zhan D C, et al. Few-shot learning via embedding adaptation with set-to-set functions[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020: 8808--8817.
[12]
Antoniou A, Edwards H, Storkey A. How to train your MAML[J]. arXiv preprint arXiv:1810.09502, 2018.
[13]
Baik S, Hong S, Lee K M. Learning to forget for meta-learning[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020: 2379--2387.
[14]
Vinyals O, Blundell C, Lillicrap T, et al. Matching networks for one shot learning[J]. Advances in neural information processing systems, 2016, 29: 3630--3638.
[15]
Bengio Y, Louradour J, Collobert R, et al. Curriculum learning[C]//Proceedings of the 26th annual international conference on machine learning. 2009: 41--48.
[16]
Wang X, Chen Y, Zhu W. A Survey on Curriculum Learning[J]. arXiv preprint arXiv:2010.13166, 2020.
[17]
Liu C, Wang Z, Sahoo D, et al. Adaptive Task Sampling for Meta-learning[C]//Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23--28, 2020, Proceedings, Part XVIII 16. Springer International Publishing, 2020: 752--769.
[18]
Oreshkin B N, Rodriguez P, Lacoste A. Tadam: Task dependent adaptive metric for improved few-shot learning[J]. arXiv preprint arXiv:1805.10123, 2018.
[19]
Zhang C, Cai Y, Lin G, et al. DeepEMD: Few-Shot Image Classification With Differentiable Earth Mover's Distance and Structured Classifiers[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2020: 12203--12213.
[20]
Vuorio R, Sun S H, Hu H, et al. Multimodal model-agnostic meta-learning via task-aware modulation[J]. arXiv preprint arXiv:1910.13616, 2019.
[21]
Patacchiola M, Turner J, Crowley E J, et al. Bayesian meta-learning for the fewshot setting via deep kernels[J]. 2020.
[22]
Baik S, Choi M, Choi J, et al. Meta-learning with adaptive hyperparameters[J]. arXiv preprint arXiv:2011.00209, 2020.
[23]
Rajeswaran A, Finn C, Kakade S, et al. Meta-learning with implicit gradients[J]. 2019.
[24]
Hacohen G, Weinshall D. On the power of curriculum learning in training deep networks[C]//International Conference on Machine Learning. PMLR, 2019: 2535--2544.
[25]
Matiisen T, Oliver A, Cohen T, et al. Teacher-student curriculum learning[J]. IEEE transactions on neural networks and learning systems, 2019, 31(9): 3732--3740.
[26]
Graves A, Bellemare M G, Menick J, et al. Automated curriculum learning for neural networks[C]//international conference on machine learning. PMLR, 2017: 1311--1320.
[27]
Kumar M, Packer B, Koller D. Self-paced learning for latent variable models[J]. Advances in neural information processing systems, 2010, 23: 1189--1197.
[28]
Shrivastava A, Gupta A, Girshick R. Training region-based object detectors with online hard example mining[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2016: 761--769.
[29]
Sun Q, Liu Y, Chua T S, et al. Meta-transfer learning for few-shot learning[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019: 403--412.
[30]
Noble W S. What is a support vector machine?[J]. Nature biotechnology, 2006, 24(12): 1565--1567.
[31]
Pedersen T, Patwardhan S, Michelizzi J. WordNet:: Similarity-Measuring the Relatedness of Concepts[C]//AAAI. 2004, 4: 25--29.
[32]
Leacock C, Chodorow M. Combining local context and WordNet similarity for word sense identification[J]. WordNet: An electronic lexical database, 1998, 49(2): 265--283.
[33]
Mishra N, Rohaninejad M, Chen X, et al. A simple neural attentive meta-learner[J]. arXiv preprint arXiv:1707.03141, 2017.
[34]
Munkhdalai T, Yuan X, Mehri S, et al. Rapid adaptation with conditionally shifted neurons[C]//International Conference on Machine Learning. PMLR, 2018: 3664--3673.
[35]
Lee K, Maji S, Ravichandran A, et al. Meta-learning with differentiable convex optimization[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019: 10657--10665.
Index Terms
- Curriculum Meta Learning: Learning to Learn from Easy to Hard
Recommendations
Curriculum-Based Meta-learning
MM '21: Proceedings of the 29th ACM International Conference on MultimediaMeta-learning offers an effective solution to learn new concepts with scarce supervision through an episodic training scheme: a series of target-like tasks sampled from base classes are sequentially fed into a meta-learner to extract common knowledge ...
Curriculum learning for reinforcement learning domains: a framework and survey
Reinforcement learning (RL) is a popular paradigm for addressing sequential decision tasks in which the agent has only limited environmental feedback. Despite many advances over the past three decades, learning in many domains still requires a large ...
Learning an explicit hyper-parameter prediction function conditioned on tasks
Meta learning has attracted much attention recently in machine learning community. Contrary to conventional machine learning aiming to learn inherent prediction rules to predict labels for new query data, meta learning aims to learn the learning ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
October 2021
1723 pages
ISBN:9781450384322
DOI:10.1145/3501409
Copyright © 2021 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 31 December 2021
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
EITCE 2021
EITCE 2021: 2021 5th International Conference on Electronic Information Technology and Computer Engineering
October 22 - 24, 2021
Xiamen, China
Acceptance Rates
EITCE '21 Paper Acceptance Rate 294 of 531 submissions, 55%;
Overall Acceptance Rate 508 of 972 submissions, 52%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 157Total Downloads
- Downloads (Last 12 months)22
- Downloads (Last 6 weeks)3
Reflects downloads up to 13 Feb 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in