Abstract
In the realm of deep learning applied to real-world scenarios, the existence of noisy labels is an inevitable factor that can detrimentally affect the models’ performance. Most state-of-the-art methods for learning from noisy labels rely on sample selection strategies that partition the training data into clean and noisy labeled samples. Subsequently, these noisy label samples are treated as unlabeled samples, and the empirical vicinal risk is minimized through semi-supervised learning. Therefore, accurately identifying noisy labels contributes to enhancing the performance of the model. To enhance the accuracy and stability of sample selection, this paper proposes utilizing the mean and variance of the loss sequence to identify clean samples and noisy ones. Nonetheless, sample selection is not entirely effective in eliminating noisy label samples, as a small fraction of them are inadvertently considered as clean samples. Consequently, we propose Weighted Neighborhood Consistency Regularization (WNCR), which alleviates the impact of residual noisy labels by encouraging the neural network to maintain consistency in its predictions with those of its k-nearest neighbors for each sample. Extensive experiments on multiple synthetic and real-world noisy datasets demonstrate that our method outperforms the state-of-the-art methods at multiple noise levels.
Supported by the National Natural Science Foundation of China under Grant No. 62272180 and No. 62272176. The computation is completed in the HPC Platfrorm of Huazhong University of Science and Technology.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Arazo, E., Ortego, D., Albert, P., O’Connor, N., McGuinness, K.: Unsupervised label noise modeling and loss correction. In: International Conference on Machine Learning, pp. 312–321. PMLR (2019)
Arpit, D., et al.: A closer look at memorization in deep networks. In: International Conference on Machine Learning, pp. 233–242. PMLR (2017)
Berthelot, D., Carlini, N., Goodfellow, I., Papernot, N., Oliver, A., Raffel, C.A.: Mixmatch: a holistic approach to semi-supervised learning. In: Advances in Neural Information Processing Systems, vol. 32 (2019)
Blum, A., Kalai, A., Wasserman, H.: Noise-tolerant learning, the parity problem, and the statistical query model. J. ACM (JACM) 50(4), 506–519 (2003)
Chen, X., Gupta, A.: Webly supervised learning of convolutional networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1431–1439 (2015)
Cheng, D., et al.: Instance-dependent label-noise learning with manifold-regularized transition matrix estimation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 16630–16639 (2022)
Cordeiro, F.R., Belagiannis, V., Reid, I., Carneiro, G.: Propmix: hard sample filtering and proportional mixup for learning with noisy labels. arXiv preprint arXiv:2110.11809 (2021)
Cordeiro, F.R., Sachdeva, R., Belagiannis, V., Reid, I., Carneiro, G.: Longremix: robust learning with high confidence samples in a noisy label environment. Pattern Recogn. 133, 109013 (2023)
Domingos, P.: A few useful things to know about machine learning. Commun. ACM 55(10), 78–87 (2012)
Ghosh, A., Kumar, H., Sastry, P.S.: Robust loss functions under label noise for deep neural networks. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 31 (2017)
Goldberger, J., Ben-Reuven, E.: Training deep neural-networks using a noise adaptation layer. In: International Conference on Learning Representations (2016)
Goldberger, J., Ben-Reuven, E.: Training deep neural-networks using a noise adaptation layer. In: International Conference on Learning Representations (2017)
Han, B., et al.: Co-teaching: robust training of deep neural networks with extremely noisy labels. In: Advances in Neural Information Processing Systems, vol. 31 (2018)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Hendrycks, D., Mazeika, M., Wilson, D., Gimpel, K.: Using trusted data to train deep networks on labels corrupted by severe noise. In: Advances in Neural Information Processing Systems, vol. 31 (2018)
Howard, J., Ruder, S.: Universal language model fine-tuning for text classification. arXiv preprint arXiv:1801.06146 (2018)
Iscen, A., Valmadre, J., Arnab, A., Schmid, C.: Learning with neighbor consistency for noisy labels. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4672–4681 (2022)
Jiang, L., Zhou, Z., Leung, T., Li, L.J., Fei-Fei, L.: Mentornet: learning data-driven curriculum for very deep neural networks on corrupted labels. In: International Conference on Machine Learning, pp. 2304–2313. PMLR (2018)
Krizhevsky, A., Hinton, G., et al.: Learning multiple layers of features from tiny images (2009)
Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, vol. 25 (2012)
Li, J., Socher, R., Hoi, S.C.: Dividemix: learning with noisy labels as semi-supervised learning. arXiv preprint arXiv:2002.07394 (2020)
Liu, S., Zhu, Z., Qu, Q., You, C.: Robust training under label noise by over-parameterization. In: International Conference on Machine Learning, pp. 14153–14172. PMLR (2022)
Liu, Y., Cheng, H., Zhang, K.: Identifiability of label noise transition matrix. In: International Conference on Machine Learning, pp. 21475–21496. PMLR (2023)
Liu, Y., Guo, H.: Peer loss functions: learning from noisy labels without knowing noise rates. In: International Conference on Machine Learning, pp. 6226–6236. PMLR (2020)
Onal, K.D., et al.: Neural information retrieval: at the end of the early years. Inf. Retrieval J. 21, 111–182 (2018)
Rong, Q., Yuan, L., Li, G., Li, J., Zhang, L., Ding, X.: A static bi-dimensional sample selection for federated learning with label noise. In: Wang, X., et al. (eds.) DASFAA 2023. LNCS, vol. 13943, pp. 735–744. Springer, Cham (2023). https://doi.org/10.1007/978-3-031-30637-2_49
Shalev-Shwartz, S., Ben-David, S.: Understanding Machine Learning: From Theory to Algorithms. Cambridge University Press, Cambridge (2014)
Smart, B., Carneiro, G.: Bootstrapping the relationship between images and their clean and noisy labels. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 5344–5354 (2023)
Sun, Z., Liu, H., Wang, Q., Zhou, T., Wu, Q., Tang, Z.: Co-LDL: a co-training-based label distribution learning method for tackling label noise. IEEE Trans. Multimedia 24, 1093–1104 (2021)
Tan, C., Xia, J., Wu, L., Li, S.Z.: Co-learning: learning from noisy labels with self-supervision. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 1405–1413 (2021)
Wang, H., Xiao, R., Dong, Y., Feng, L., Zhao, J.: Promix: combating label noise via maximizing clean sample utility. arXiv preprint arXiv:2207.10276 (2022)
Wang, Y., Ma, X., Chen, Z., Luo, Y., Yi, J., Bailey, J.: Symmetric cross entropy for robust learning with noisy labels. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 322–330 (2019)
Wei, H., Feng, L., Chen, X., An, B.: Combating noisy labels by agreement: a joint training method with co-regularization. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13726–13735 (2020)
Wei, H., et al.: Logit clipping for robust learning against label noise. arXiv preprint arXiv:2212.04055 (2022)
Wei, J., Zhu, Z., Cheng, H., Liu, T., Niu, G., Liu, Y.: Learning with noisy labels revisited: a study using real-world human annotations. In: International Conference on Learning Representations (2022). https://openreview.net/forum?id=TBWA6PLJZQm
Xia, X., et al.: Are anchor points really indispensable in label-noise learning? In: Advances in Neural Information Processing Systems, vol. 32 (2019)
Xiao, T., Xia, T., Yang, Y., Huang, C., Wang, X.: Learning from massive noisy labeled data for image classification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2691–2699 (2015)
Yi, K., Wu, J.: Probabilistic end-to-end noise correction for learning with noisy labels. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7017–7025 (2019)
Yu, X., Han, B., Yao, J., Niu, G., Tsang, I., Sugiyama, M.: How does disagreement help generalization against label corruption? In: International Conference on Machine Learning, pp. 7164–7173. PMLR (2019)
Zhang, C., Bengio, S., Hardt, M., Recht, B., Vinyals, O.: Understanding deep learning (still) requires rethinking generalization. Commun. ACM 64(3), 107–115 (2021)
Zhang, Y., Niu, G., Sugiyama, M.: Learning noise transition matrix from only noisy labels via total variation regularization. In: International Conference on Machine Learning, pp. 12501–12512. PMLR (2021)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Rong, Q., Zhang, L., Yuan, L., Ding, X., Li, G. (2024). Noisy Label Learning Based on Weighted Neighborhood Consistency. In: Zhang, W., Tung, A., Zheng, Z., Yang, Z., Wang, X., Guo, H. (eds) Web and Big Data. APWeb-WAIM 2024. Lecture Notes in Computer Science, vol 14965. Springer, Singapore. https://doi.org/10.1007/978-981-97-7244-5_4
Download citation
DOI: https://doi.org/10.1007/978-981-97-7244-5_4
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-97-7243-8
Online ISBN: 978-981-97-7244-5
eBook Packages: Computer ScienceComputer Science (R0)