Abstract
Noisy data samples is a common problem for deep learning models applied to real-world applications. In this context, noisy samples refer to samples with incorrect labels, which can potentially degenerate the robustness of a model. Several works account for this issue in multi-class scenarios. However, despite a number of possible applications, multi-label noise remains an under-explored research field. In this work, two novel approaches to handle noise in this scenario are presented. First, we propose a new multi-label version of the Small Loss Approach (SLA), formerly multi-class, to handle multi-label noise. Second, we apply the multi-label SLA to a novel model, Learning by SLA Multi-label, based on Co-teaching. The proposed model achieves a performance gain of \(15\%\) in the benchmark UcMerced when compared to its baseline Co-teaching and a standard model (without any noise-handling technique). In addition, the model is also evaluated in a real-world scenario of underwater equipment imagery classification, yielding a relative improvement of \(9\%\) in F1-Score.
Supported by organization Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Ahmadi, Z., Kramer, S.: A label compression method for online multi-label classification. Pattern Recogn. Lett. 111, 64–71 (2018)
Arpit, D., et al.: A closer look at memorization in deep networks. In: International conference on machine learning, pp. 233–242. PMLR (2017)
Burgert, T., Ravanbakhsh, M., Demir, B.: On the effects of different types of label noise in multi-label remote sensing image classification. IEEE Trans. Geosci. Remote Sens. 60, 1–13 (2022)
Chaudhuri, B., Demir, B., Chaudhuri, S., Bruzzone, L.: Multilabel remote sensing image retrieval using a semisupervised graph-theoretic method. IEEE Trans. Geosci. Remote Sens. 56(2), 1144–1158 (2017)
Chen, Z.M., Wei, X.S., Wang, P., Guo, Y.: Multi-label image recognition with graph convolutional networks. In: Proceedings of the IEEE/CVF Conference On Computer Vision And Pattern Recognition, pp. 5177–5186 (2019)
Chicco, D., Jurman, G.: The advantages of the matthews correlation coefficient (mcc) over f1 score and accuracy in binary classification evaluation. BMC Genom. 21(1), 1–13 (2020)
Coulibaly, S., Kamsu-Foguem, B., Kamissoko, D., Traore, D.: Deep convolution neural network sharing for the multi-label images classification. Mach. Learn. Appl. 10, 100422 (2022)
Cuevas, A., Febrero, M., Fraiman, R.: An anova test for functional data. Comput. Stat. Data Anal. 47(1), 111–122 (2004)
Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference On Computer Vision And Pattern Recognition, pp. 248–255. IEEE (2009)
Deng, L.: The mnist database of handwritten digit images for machine learning research [best of the web]. IEEE Signal Process. Mag. 29(6), 141–142 (2012)
Ghosh, A., Kumar, H., Sastry, P.S.: Robust loss functions under label noise for deep neural networks. In: Proceedings of the AAAI Conference On Artificial Intelligence, vol. 31 (2017)
Han, B., et al.: Co-teaching: Robust training of deep neural networks with extremely noisy labels. In: Advances in Neural Information Processing Systems, vol. 31 (2018)
Hu, M., Han, H., Shan, S., Chen, X.: Multi-label learning from noisy labels with non-linear feature transformation. In: Computer Vision-ACCV 2018: 14th Asian Conference on Computer Vision, Perth, Australia, December 2–6, 2018, Revised Selected Papers, Part V 14. pp. 404–419. Springer (2019)
Hu, M., Han, H., Shan, S., Chen, X.: Weakly supervised image classification through noise regularization. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11517–11525 (2019)
Huang, L., Zhang, C., Zhang, H.: Self-adaptive training: bridging the supervised and self-supervised learning. arXiv preprint arXiv:2101.08732 (2021)
Inoue, N., Simo-Serra, E., Yamasaki, T., Ishikawa, H.: Multi-label fashion image classification with minimal human supervision. In: Proceedings of the IEEE International Conference on Computer Vision Workshops, pp. 2261–2267 (2017)
Jiang, L., Zhou, Z., Leung, T., Li, L.J., Fei-Fei, L.: Mentornet: Learning data-driven curriculum for very deep neural networks on corrupted labels. In: International Conference On Machine Learning, pp. 2304–2313. PMLR (2018)
Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
Krizhevsky, A., Hinton, G., et al.: Learning multiple layers of features from tiny images (2009)
Liu, S., Niles-Weed, J., Razavian, N., Fernandez-Granda, C.: Early-learning regularization prevents memorization of noisy labels. Adv. Neural. Inf. Process. Syst. 33, 20331–20342 (2020)
Liu, T., Tao, D.: Classification with noisy labels by importance reweighting. IEEE Trans. Pattern Anal. Mach. Intell. 38(3), 447–461 (2015)
Liu, W., Jiang, Y.G., Luo, J., Chang, S.F.: Noise resistant graph ranking for improved web image search. In: CVPR 2011, pp. 849–856. IEEE (2011)
Malach, E., Shalev-Shwartz, S.: Decoupling “when to update” from “how to update". In: Advances in Neural Information Processing Systems, vol. 30 (2017)
Sanderson, T., Scott, C.: Class proportion estimation with application to multiclass anomaly rejection. In: Artificial Intelligence and Statistics, pp. 850–858. PMLR (2014)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)
Song, H., Kim, M., Lee, J.G.: Selfie: Refurbishing unclean samples for robust deep learning. In: International Conference on Machine Learning, pp. 5907–5915. PMLR (2019)
Van Rooyen, B., Williamson, R.C.: A theory of learning with corrupted labels. J. Mach. Learn. Res. 18(1), 8501–8550 (2017)
Veit, A., Alldrin, N., Chechik, G., Krasin, I., Gupta, A., Belongie, S.: Learning from noisy large-scale datasets with minimal supervision. In: Proceedings of the IEEE Conference On Computer Vision And Pattern Recognition, pp. 839–847 (2017)
Wei, H., Feng, L., Chen, X., An, B.: Combating noisy labels by agreement: A joint training method with co-regularization. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13726–13735 (2020)
Wei, T., Shi, J.X., Tu, W.W., Li, Y.F.: Robust long-tailed learning under label noise. arXiv preprint arXiv:2108.11569 (2021)
Welinder, P., Branson, S., Perona, P., Belongie, S.: The multidimensional wisdom of crowds. In: Advances in Neural Information Processing Systems, vol. 23 (2010)
Xiao, T., Xia, T., Yang, Y., Huang, C., Wang, X.: Learning from massive noisy labeled data for image classification. In: CVPR (2015)
Yang, Y., Newsam, S.: Bag-of-visual-words and spatial extensions for land-use classification. In: Proceedings of the 18th SIGSPATIAL International Conference on Advances in Geographic Information Systems, pp. 270–279 (2010)
Yao, L., Poblenz, E., Dagunts, D., Covington, B., Bernard, D., Lyman, K.: Learning to diagnose from scratch by exploiting dependencies among labels. arXiv preprint arXiv:1710.10501 (2017)
Yao, Y., Sun, Z., Zhang, C., Shen, F., Wu, Q., Zhang, J., Tang, Z.: Jo-src: A contrastive approach for combating noisy labels. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5192–5201 (2021)
Yu, X., Han, B., Yao, J., Niu, G., Tsang, I., Sugiyama, M.: How does disagreement help generalization against label corruption? In: International Conference on Machine Learning, pp. 7164–7173. PMLR (2019)
Acknowledgements
The authors would like to thank Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq), Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (Capes) and Pontifícia Universidade Católica do Rio de Janeiro (PUC-Rio) for their financial support.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Sousa, V., Pereira, A.L., Kohler, M., Pacheco, M. (2023). Learning by Small Loss Approach Multi-label to Deal with Noisy Labels. In: Gervasi, O., et al. Computational Science and Its Applications – ICCSA 2023. ICCSA 2023. Lecture Notes in Computer Science, vol 13956 . Springer, Cham. https://doi.org/10.1007/978-3-031-36805-9_26
Download citation
DOI: https://doi.org/10.1007/978-3-031-36805-9_26
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-36804-2
Online ISBN: 978-3-031-36805-9
eBook Packages: Computer ScienceComputer Science (R0)