Abstract
Recently, Few-Shot Learning (FSL), or learning from very few (typically 1 or 5) examples per novel class (unseen during training), has received a lot of attention and significant performance advances. While number of techniques have been proposed for FSL, several factors have emerged as most important for FSL performance, awarding SOTA even to the simplest of techniques. These are: the backbone architecture (bigger is better), type of pre-training (meta-training vs multi-class), quantity and diversity of the base classes (the more the merrier), and using auxiliary self-supervised tasks (a proxy for increasing the diversity). In this paper we propose TAFSSL, a simple technique for improving the few shot performance in cases when some additional unlabeled data accompanies the few-shot task. TAFSSL is built upon the intuition of reducing the feature and sampling noise inherent to few-shot tasks comprised of novel classes unseen during pre-training. Specifically, we show that on the challenging miniImageNet and tieredImageNet benchmarks, TAFSSL can improve the current state-of-the-art in both transductive and semi-supervised FSL settings by more than \(5\%\), while increasing the benefit of using unlabeled data in FSL to above \(10\%\) performance gain.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Alfassy, A., et al.: LaSO: Label-Set Operations networks for multi-label few-shot learning. In: CVPR (2019)
Antoniou, A., Storkey, A., Edwards, H.: Data Augmentation Generative Adversarial Networks. arXiv:1711.04340 (2017). https://arxiv.org/pdf/1711.04340.pdf
Chen, W.Y., Liu, Y.C., Kira, Z., Wang, Y.C., Huang, J.B.: A closer look at few-shot classification. In: ICLR (2019)
Chen, Z., Fu, Y., Zhang, Y., Jiang, Y.G., Xue, X., Sigal, L.: Multi-level semantic feature augmentation for one-shot learning. IEEE Trans. Image Process. 28(9), 4594–4605 (2019). https://doi.org/10.1109/tip.2019.2910052
Comon, P.: Independent component analysis, A new concept? Technical Report (1994)
Cubuk, E.D., Zoph, B., Mané, D., Vasudevan, V., Le, Q.V.: AutoAugment: Learning Augmentation Policies from Data. https://arxiv.org/pdf/1805.09501v1.pdf
Devos, A., Grossglauser, M.: Subspace Networks for Few-shot Classification. Technical Report (2019)
Dhillon, G.S., Chaudhari, P., Ravichandran, A., Soatto, S.: A Baseline For Few-Shot Image Classification. Technical Report (2019)
Dosovitskiy, A., Springenberg, J.T., Tatarchenko, M., Brox, T.: Learning to generate chairs, tables and cars with convolutional networks. IEEE Trans. Pattern Anal. Mach. Intell. 39(4), 692–705 (2017). https://doi.org/10.1109/TPAMI.2016.2567384
Doveh, S., et al.: MetAdapt: Meta-Learned Task-Adaptive Architecture for Few-Shot Classification. Technical Report (2019)
Dvornik, N., Schmid, C., Mairal, J.: Diversity with cooperation: ensemble methods for few-shot classification. In: The IEEE International Conference on Computer Vision (ICCV) (2019). http://arxiv.org/abs/1903.11341
Finn, C., Abbeel, P., Levine, S.: Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks. arXiv:1703.03400 (2017). http://arxiv.org/abs/1703.03400
Fukunaga, K., Hostetler, L.D.: The estimation of the gradient of a density function, with applications in pattern recognition. IEEE Trans. Inf. Theory 21(1), 32–40 (1975). https://doi.org/10.1109/TIT.1975.1055330
Garcia, V., Bruna, J.: Few-Shot Learning with Graph Neural Networks, pp. 1–13. arXiv:1711.04043 (2017). http://arxiv.org/abs/1711.04043
Gidaris, S., Bursuc, A., Komodakis, N., Pérez, P., Cord, M.: Boosting Few-Shot Visual Learning with Self-Supervision, 6 2019. http://arxiv.org/abs/1906.05186
Guu, K., Hashimoto, T.B., Oren, Y., Liang, P.: Generating Sentences by Editing Prototypes. Arxiv:1709.08878 (2017). https://arxiv.org/pdf/1709.08878.pdf
Hariharan, B., Girshick, R.: Low-shot visual recognition by shrinking and hallucinating features. In: IEEE International Conference on Computer Vision (ICCV) (2017). https://arxiv.org/pdf/1606.02819.pdf
He, K., Zhang, X., Ren, S., Sun, J.: Deep Residual Learning for Image Recognition. arXiv:1512.03385 (2015). https://arxiv.org/pdf/1512.03385.pdf
Huang, G., Liu, Z., Maaten, V.D.L., Weinberger, K.Q.: Densely connected convolutional networks. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) pp. 2261–2269 (2017). https://doi.org/10.1109/CVPR.2017.243, https://arxiv.org/pdf/1608.06993.pdf
Jiang, X., Havaei, M., Varno, F., Chartrand, G.: Learning To Learn With Conditional Class Dependencies, pp. 1–11 (2019)
Kim, J., Kim, T., Kim, S., Yoo, C.D.: Edge-Labeling Graph Neural Network for Few-shot Learning. Technical Report
Lee, K., Maji, S., Ravichandran, A., Soatto, S.: Meta-Learning with Differentiable Convex Optimization. In: CVPR (2019). https://github.com/kjunelee/MetaOptNet
Li, H., Eigen, D., Dodge, S., Zeiler, M., Wang, X.: Finding Task-Relevant Features for Few-Shot Learning by Category Traversal, vol. 1 (2019). http://arxiv.org/abs/1905.11116
Li, X., et al.: Learning to Self-Train for Semi-Supervised Few-Shot Classification, 6 2019. http://arxiv.org/abs/1906.00562
Li, Z., Zhou, F., Chen, F., Li, H.: Meta-SGD: Learning to Learn Quickly for Few-Shot Learning. arXiv:1707.09835 (2017). http://arxiv.org/abs/1707.09835
Lim, S., Kim, I., Kim, T., Kim, C., Brain, K., Kim, S.: Fast AutoAugment. Technical Report (2019)
Lin, T.-Y., et al.: Microsoft COCO: common objects in context. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8693, pp. 740–755. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10602-1_48
Liu, Y., et al.: Learning To Propagate Labels: Transductive Propagation Networ For Few-Shot Learning (2019)
Lloyd, S.P., Lloyd, S.P.: Least squares quantization in PCM. IEEE Trans. Inf. Theory, 28, 129–137 (1982). https://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.131.1338
Munkhdalai, T., Yu, H.: Meta Networks. arXiv:1703.00837 (2017). https://doi.org/10.1093/mnrasl/slx008, http://arxiv.org/abs/1703.00837
Nakamura, A., Harada, T.: Revisiting Fine-Tuning for Few-Shot Learning. Technical Report
Oreshkin, B.N., Rodriguez, P., Lacoste, A.: TADAM: task dependent adaptive metric for improved few-shot learning. NeurIPS, 5 2018. http://arxiv.org/abs/1805.10123
Park, D., Ramanan, D.: Articulated pose estimation with tiny synthetic videos. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 58–66, October 2015. https://doi.org/10.1109/CVPRW.2015.7301337
Pearson, K.: On lines and planes of closest fit to systems of points in space. Lond. Edinb. Dublin Philos. Mag. J. Sci. 2(11), 559–572 (1901). https://doi.org/10.1080/14786440109462720
Qiao, L., Shi, Y., Li, J., Wang, Y., Huang, T., Tian, Y.: Transductive Episodic-Wise Adaptive Metric for Few-Shot Learning (2019). http://arxiv.org/abs/1910.02224
Ravi, S., Larochelle, H.: Optimization as a model for few-shot learning. In: ICLR, pp. 1–11 (2017). https://openreview.net/pdf?id=rJY0-Kcll
Reed, S., et al.: Few-shot autoregressive density estimation: towards learning to learn distributions, pp. 1–11 (2018). arXiv:1710.10304 (2016)
Ren, M., et al.: Meta-learning for semi-supervised few-shot classification. In: ICLR, 3 2018. http://arxiv.org/abs/1803.00676, http://bair.berkeley.edu/blog/2017/07/18/
Rippel, O., Paluri, M., Dollar, P., Bourdev, L.: Metric Learning with Adaptive Density Discrimination, pp. 1–15. arXiv:1511.05939 (2015). http://arxiv.org/abs/1511.05939
Russakovsky, O., et al.: ImageNet large scale visual recognition challenge. In: IJCV, 9 2015. http://arxiv.org/abs/1409.0575
Rusu, A.A., et al.: Meta-learning with latent embedding optimization. In: ICLR, 7 2018. http://arxiv.org/abs/1807.05960
Saito, K., Kim, D., Sclaroff, S., Darrell, T., Saenko, K.: Semi-supervised domain adaptation via minimax entropy. In: ICCV, 4 2019. http://arxiv.org/abs/1904.06487
Santoro, A., Bartunov, S., Botvinick, M., Wierstra, D., Lillicrap, T.: Meta-learning with memory-augmented neural networks. J. Mach. Learn. Res. (Proceedings of The 33rd International Conference on Machine Learning), vol. 48, pp. 1842–1850 (2016). https://doi.org/10.1002/2014GB005021
Schwartz, E., Karlinsky, L., Feris, R., Giryes, R., Bronstein, A.M.: Baby steps towards few-shot learning with multiple semantics, pp. 1–11 (2019). http://arxiv.org/abs/1906.01905
Schwartz, E., et al.: Delta-Encoder: an effective sample synthesis method for few-shot object recognition. NeurIPS (2018). https://arxiv.org/pdf/1806.04734.pdf
Simon, C., Koniusz, P., Harandi, M.: Projective sub-space networks for few-shot learning. In: ICLR 2019 OpenReview. https://openreview.net/pdf?id=rkzfuiA9F7
Snell, J., Swersky, K., Zemel, R.: Prototypical networks for few-shot learning. In: NIPS (2017). http://arxiv.org/abs/1703.05175
Su, H., Qi, C.R., Li, Y., Guibas, L.J.: Render for CNN viewpoint estimation in images using CNNs trained with rendered 3D model views.pdf. In: IEEE International Conference on Computer Vision (ICCV), pp. 2686–2694 (2015)
Sung, F., Yang, Y., Zhang, L., Xiang, T., Torr, P.H.S., Hospedales, T.M.: Learning to Compare: Relation Network for Few-Shot Learning. https://arxiv.org/pdf/1711.06025.pdf
Sung, F., Yang, Y., Zhang, L., Xiang, T., Torr, P.H.S., Hospedales, T.M.: Learning to Compare: Relation Network for Few-Shot Learning. arXiv:1711.06025 (2017), http://arxiv.org/abs/1711.06025
Vinyals, O., Blundell, C., Lillicrap, T., Kavukcuoglu, K., Wierstra, D.:Matching networks for one shot learning. In: NIPS (2016).https://doi.org/10.1109/CVPR.2016.95, http://arxiv.org/abs/1606.04080
Wang, Y., Chao, W.L., Weinberger, K.Q., van der Maaten, L.: SimpleShot: Revisiting Nearest-Neighbor Classification for Few-Shot Learning, 11 2019. http://arxiv.org/abs/1911.04623
Wang, Y.X., Girshick, R., Hebert, M., Hariharan, B.: Low-Shot Learning from Imaginary Data. arXiv:1801.05401 (2018). http://arxiv.org/abs/1801.05401
Weinberger, K.Q., Saul, L.K.: Distance metric learning for large margin nearest neighbor classification. J. Mach. Learn. Res. 10, 207–244 (2009). https://doi.org/10.1126/science.277.5323.215
Xing, C., Rostamzadeh, N., Oreshkin, B.N., Pinheiro, P.O.: Adaptive Cross-Modal Few-Shot Learning (2019). https://arxiv.org/pdf/1902.07104.pdf, http://arxiv.org/abs/1902.07104
Yu, A., Grauman, K.: Semantic jitter: dense supervision for visual comparisons via synthetic images. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 5571–5580, October 2017. https://doi.org/10.1109/ICCV.2017.594
Zhang, J., Zhao, C., Ni, B., Xu, M., Yang, X.: Variational few-shot learning. In: IEEE International Conference on Computer Vision (ICCV) (2019)
Zhou, F., Wu, B., Li, Z.: Deep Meta-Learning: Learning to Learn in the Concept Space. Technical Report, 2 2018. http://arxiv.org/abs/1802.03596
Acknowledgment
This material is based upon work supported by the Defense Advanced Research Projects Agency (DARPA) under Contract No. FA8750-19-C-1001. Any opinions, ndings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reect the views of DARPA. Raja Giryes is supported by ERC-StG grant no. 757497 (SPADE).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Lichtenstein, M., Sattigeri, P., Feris, R., Giryes, R., Karlinsky, L. (2020). TAFSSL: Task-Adaptive Feature Sub-Space Learning for Few-Shot Classification. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, JM. (eds) Computer Vision – ECCV 2020. ECCV 2020. Lecture Notes in Computer Science(), vol 12352. Springer, Cham. https://doi.org/10.1007/978-3-030-58571-6_31
Download citation
DOI: https://doi.org/10.1007/978-3-030-58571-6_31
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-58570-9
Online ISBN: 978-3-030-58571-6
eBook Packages: Computer ScienceComputer Science (R0)