Abstract
Semi-supervised learning (SSL) has received significant attention due to its ability to use limited labeled data and various unlabeled data to train models with high generalization performance. However, the assumption of a balanced class distribution in traditional SSL approaches limits a wide range of real applications, where the training data exhibits long-tailed distributions. As a consequence, the model is biased towards head classes and disregards tail classes, thereby leading to severe class-aware bias. Additionally, since the unlabeled data may contain out-of-distribution (OOD) samples without manual filtering, the model will be inclined to assign OOD samples to non-tail classes with high confidence, which further overwhelms the tail classes. To alleviate this class-aware bias, we propose an end-to-end semi-supervised method Debias Class-Aware Bias (DeCAB). DeCAB introduces positive-pair scores for contrastive learning instead of positive-negative pairs based on unreliable pseudo-labels, avoiding false negative pairs negatively impacts the feature space. At the same time, DeCAB utilizes class-aware thresholds to select more tail samples and selective sample reweighting for feature learning, preventing OOD samples from being misclassified as head classes and accelerating the convergence speed of the model. Experimental results demonstrate that DeCAB is robust in various semi-supervised benchmarks and achieves state-of-the-art performance. Our code is temporarily available at https://github.com/xlhuang132/decab.
This work was supported in part by the National Natural Science Foundation of China under Grants 62002302, 62306181; in part by the FuXiaQuan National Independent Innovation Demonstration Zone Collaborative Innovation Platform under Grant 3502ZCQXT2022008; in part by the China Fundamental Research Funds for the Central Universities under Grants 20720230038.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Berthelot, D., Carlini, N., Goodfellow, I., Papernot, N., Oliver, A., Raffel, C.A.: MixMatch: a holistic approach to semi-supervised learning. In: Advances in Neural Information Processing Systems, pp. 5050–5060 (2019)
Chen, Y., Zhu, X., Li, W., Gong, S.: Semi-supervised learning under class distribution mismatch. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp. 3569–3576 (2020)
Cubuk, E.D., Zoph, B., Mane, D., Vasudevan, V., Le, Q.V.: AutoAugment: learning augmentation policies from data. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 113–123 (2019)
Cubuk, E.D., Zoph, B., Shlens, J., Le, Q.: RandAugment: practical automated data augmentation with a reduced search space. In: Advances in Neural Information Processing Systems, pp. 18613–18624 (2020)
Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: ImageNet: a large-scale hierarchical image database. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009)
Fan, Y., Dai, D., Kukleva, A., Schiele, B.: CoSSL: co-learning of representation and classifier for imbalanced semi-supervised learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14574–14584 (2022)
Guo, L.Z., Zhang, Z.Y., Jiang, Y., Li, Y.F., Zhou, Z.H.: Safe deep semi-supervised learning for unseen-class unlabeled data. In: International Conference on Machine Learning, pp. 3897–3906 (2020)
Hestness, J., et al.: Deep learning scaling is predictable, empirically. arXiv preprint arXiv:1712.00409 (2017)
Jozefowicz, R., Vinyals, O., Schuster, M., Shazeer, N., Wu, Y.: Exploring the limits of language modeling. arXiv preprint arXiv:1602.02410 (2016)
Ke, Z., Wang, D., Yan, Q., Ren, J., Lau, R.W.: Dual student: Breaking the limits of the teacher in semi-supervised learning. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 6728–6736 (2019)
Krizhevsky, A., Hinton, G., et al.: Learning multiple layers of features from tiny images. Tech. rep. (2009)
Lee, D.H., et al.: Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks. In: Workshop on Challenges in Representation Learning, ICML (2013)
Li, T., et al.: Targeted supervised contrastive learning for long-tailed recognition. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6918–6928 (2022)
Oh, Y., Kim, D.J., Kweon, I.S.: DASO: Distribution-aware semantics-oriented pseudo-label for imbalanced semi-supervised learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,786–9796 (2022)
Raffel, C., et al.: Exploring the limits of transfer learning with a unified text-to-text transformer. J. Mach. Learn. Res. 5485–5551 (2020)
Saito, K., Kim, D., Saenko, K.: OpenMatch: open-set consistency regularization for semi-supervised learning with outliers. arXiv preprint arXiv:2105.14148 (2021)
Sohn, K., et al.: FixMatch: simplifying semi-supervised learning with consistency and confidence. In: Advances in Neural Information Processing Systems, pp. 596–608 (2020)
Tarvainen, A., Valpola, H.: Mean teachers are better role models: weight-averaged consistency targets improve semi-supervised deep learning results. In: Advances in Neural Information Processing Systems, pp. 1195–1204 (2017)
Van Engelen, J.E., Hoos, H.H.: A survey on semi-supervised learning. Mach. Learn. 373–440 (2020)
Wang, H., et al.: Partial and asymmetric contrastive learning for out-of-distribution detection in long-tailed recognition. In: International Conference on Machine Learning, pp. 23446–23458 (2022)
Wei, C., Sohn, K., Mellina, C., Yuille, A., Yang, F.: CReST: a class-rebalancing self-training framework for imbalanced semi-supervised learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10857–10866 (2021)
Xu, Y., et al.: Dash: semi-supervised learning with dynamic thresholding. In: International Conference on Machine Learning, pp. 11525–11536 (2021)
Yang, F., et al.: Class-aware contrastive semi-supervised learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14421–14430 (2022)
Yang, L., Jiang, H., Song, Q., Guo, J.: A survey on long-tailed visual recognition. Int. J. Comput. Vis. 1837–1872 (2022)
Yu, F., Seff, A., Zhang, Y., Song, S., Funkhouser, T., Xiao, J.: LSUN: construction of a large-scale image dataset using deep learning with humans in the loop. arXiv preprint arXiv:1506.03365 (2015)
Yu, Q., Ikami, D., Irie, G., Aizawa, K.: Multi-task curriculum framework for open-set semi-supervised learning. In: European Conference on Computer Vision, pp. 438–454 (2020)
Zhu, J., Wang, Z., Chen, J., Chen, Y.P.P., Jiang, Y.G.: Balanced contrastive learning for long-tailed visual recognition. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6908–6917 (2022)
Zhu, X.J.: Semi-supervised learning literature survey (2005)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Appendices
Appendix
A Analysis of the Effect of OOD Data to SSL Methods
Confusion matrices of SSL methods on the testing set of CIFAR-10 under two scenarios. The training set of CIFAR-10 is utilized as the labeled and unlabeled ID data with an imbalance factor of 100, and the testing set of Tiny ImageNet (TIN) is used as OOD data. (a) shows the case without OOD data, while (b) shows the case with OOD data.
To reveal the fact that the OOD samples exacerbate the long-tail problem in existing SSL methods, we conduct a quick experiment. In detail, we consider a scenario in which the labeled data is characterized by a long-tailed distribution, with a small number of classes containing a disproportionate number of samples, while the vast majority of classes have a limited number of samples. In addition, the unlabeled data comprises both ID and OOD samples. The unlabeled ID samples follow the same class distribution and a similar long-tailed distribution to the labeled data, while the OOD samples do not belong to any of the ID classes. We conduct a quick experiment to demonstrate that the performance of existing SSL methods deteriorates when confronted with OOD data. To evaluate the model performance under various scenarios, we manipulate the imbalanced factor as well as the inclusion of OOD samples to simulate different settings. Figure 4 compares the confusion matrices of SSL methods on imbalanced training data with and without OOD data. The ID data uses the training set of CIFAR-100 with an imbalance factor of 100, while OOD data uses the testing set of Tiny ImageNet (TIN). It can be seen from the figure that the long-tailed problem leads to performance degradation on the tail class because many tail class samples are misclassified as a head class. The presence of OOD samples exacerbates the long-tailes problem for existing SSL methods.
B Algorithm Flowchart
The algorithm of DeCAB is shown in Algorithm 1.
C Visualized Comparison
In order to evaluate the learning of the model on the feature space, we perform a visualized comparative analysis of the test set features extracted from the backbone of the model obtained by each method. Figure 5 shows the t-SNE visualization of feature space about the testing set of CIFAR-10, where the model is trained on CIFAR-10-LT (IF=100, TIN). In the figure, the black circle circles the space of the easily confused head and tail classes. The feature space that is learned by other methods typically shows an aggregation of similar head and tail classes, with a significant proportion of misclassified tail classes located in the middle of the black circles. In contrast, DeCAB exhibits a much clearer separation between head and tail classes in the feature space, resulting in fewer samples being misclassified in the middle. These results illustrate that our method can obtain a better feature space.
The t-SNE visualization of feature space of CIFAR-10-LT test set, trained on CIFAR-10-LT with IF=100 and Tiny ImageNet as OOD data. The black circle circles the easily confused head and tail samples.
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Huang, X., Li, M., Lu, Y., Wang, H. (2024). DeCAB: Debiased Semi-supervised Learning for Imbalanced Open-Set Data. In: Liu, Q., et al. Pattern Recognition and Computer Vision. PRCV 2023. Lecture Notes in Computer Science, vol 14433. Springer, Singapore. https://doi.org/10.1007/978-981-99-8546-3_9
Download citation
DOI: https://doi.org/10.1007/978-981-99-8546-3_9
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-8545-6
Online ISBN: 978-981-99-8546-3
eBook Packages: Computer ScienceComputer Science (R0)