research-article

Complementary Label Queries for Efficient Active Learning

Authors:

Yunqing MaoAuthors Info & Claims

ICIGP '23: Proceedings of the 2023 6th International Conference on Image and Graphics Processing

Pages 1 - 7

https://doi.org/10.1145/3582649.3582667

Published: 07 April 2023 Publication History

Abstract

Many active learning methods are based on the assumption that a learner simply asks for the true labels of some training data from annotators. Unfortunately, it is expensive to exactly annotate instances in real-world classification tasks. To mitigate this problem, we propose a novel active learning setting, named active learning with complementary labels (ALCL). The ALCL learners only ask yes/no questions in some classes. After receiving answers from annotators, ALCL learners get a few supervised data and more training instances with complementary labels, which only specify one of the classes to which the pattern does not belong. There are two challenging issues in ALCL: the first is how to learn from these complementary labels, and another is how to sample instances to be queried. For the first issue, we propose a novel method that redistributes the weights of instances based on the balance of category contribution to learn from ordinary labels and complementary labels. For the second issue, we propose a weighting mechanism to improve existing uncertainty-based sampling strategies under this novel setup. Experiments on various datasets demonstrate the feasibility of ALCL and the superiority of our approaches.

References

[1]

Z.-H. Zhou. A brief introduction to weakly supervised learning. National science review, pages 44–53, 2018.

[2]

D. Yoo and I. S. Kweon. Learning loss for active learning. In CVPR, pages 93–102, 2019.

[3]

S. Sinha, S. Ebrahimi, and T. Darrell. Variational adversarial active learning. In ICCV, pages 5972–5981, 2019.

[4]

T. Zhang and Z.-H. Zhou. Semi-supervised optimal margin distribution machines. In IJCAI, pages 3104–3110, 2018.

[5]

H. Wang, W. Liu, Y. Zhao, C. Zhang, T. Hu, and G. Chen. Discriminative and correlative partial multi-label learning. In IJCAI, pages 3691–3697, 2019.

[6]

T. Ishida, G. Niu, W. Hu, and M. Sugiyama. Learning from complementary labels. In NeurIPS, pages 5639–5649, 2017.

[7]

T. Ishida, G. Niu, A. Menon, and M. Sugiyama. Complementary-label learning for arbitrary losses and models. In ICML, pages 2971–2980, 2019.

[8]

L. Feng, T. Kaneko, B. Han, G. Niu, B. An, and M. Sugiyama. Learning with multiple complementary labels. In ICML, pages 3072–3081, 2020.

[9]

P. Ren, Y. Xiao, X. Chang, P.-Y. Huang, Z. Li, B. B. Gupta, X. Chen, and X. Wang. A survey of deep active learning. ACM Computing Surveys (CSUR), pages 1–40, 2021.

[10]

B. Settles. From theories to queries: Active learning in practice. In active learning and experimental design workshop in conjunction with AISTATS 2010, pages 1–18, 2011.

[11]

T. Kaneko, I. Sato, and M. Sugiyama. Online multiclass classification based on prediction margin for partial feedback. arXiv preprint arXiv:1902.01056, 2019.

[12]

M. Rezaei, H. Yang, and C. Meinel. Recurrent generative adversarial network for learning imbalanced medical image semantic segmentation. Multimedia Tools and Applications, pages 15329–15348, 2020.

Digital Library

[13]

B. Settles. Active learning literature survey. 2009.

[14]

W. H. Beluch, T. Genewein, A. Nürnberger, and J. M. Köhler. The power of ensembles for active learning in image classification. In CVPR, pages 9368–9377, 2018.

[15]

Y. Gal, R. Islam, and Z. Ghahramani. Deep bayesian active learning with image data. In ICML, pages 1183–1192, 2017.

[16]

Y. Guo. Active instance sampling via matrix partition. In NIPS, pages 802–810, 2010.

[17]

K. Kim, D. Park, K. I. Kim In, and S. Y. Chun. Task-aware variational adversarial active learning. In CVPR, pages 8166–8175, 2021.

[18]

P. Hu, Z. C. Lipton, A. Anandkumar, and D. Ramanan. Active learning with partial feedback. In ICLR, 2019.

[19]

J. Gonsior, M. Thiele, and W. Lehner. Weakal: Combining active learning and weak supervision. In International Conference on Discovery Science, pages 34–49, 2020.

Digital Library

[20]

C. Zhang and K. Chaudhuri. Active learning from weak and strong labelers. In NeurIPS, pages 703–711, 2015.

[21]

T. Younesian, D. Epema, and L.-Y. Chen. Active learning for noisy data streams using weak and strong labelers. arXiv preprint arXiv:2010.14149, 2020.

[22]

S. Yan, K. Chaudhuri, and T. Javidi. Active learning from imperfect labelers. In NeurIPS, pages 2128–2136, 2016.

[23]

X. Yu, T. Liu, M. Gong, and D. Tao. Learning with biased complementary labels. In ECCV, pages 68–83, 2018.

Digital Library

[24]

A. Culotta and A. McCallum. Reducing labeling effort for structured prediction tasks. In AAAI, pages 746–751, 2005.

[25]

T. Scheffer, C. Decomain, and S. Wrobel. Active hidden markov models for information extraction. In International Symposium on Intelligent Data Analysis, pages 309–318, 2001.

[26]

Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, pages 2278–2324, 1998.

[27]

H. Xiao, K. Rasul, and R. Vollgraf. Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms. arXiv preprint arXiv:1708.07747, 2017.

[28]

T. Clanuwat, M. Bober-Irizar, A. Kitamoto, A. Lamb, K. Yamamoto, and D. Ha. Deep learning for classical japanese literature. arXiv preprint arXiv:1812.01718, 2018.

[29]

A. Krizhevsky and G. Hinton. Learning multiple layers of features from tiny images. 2009.

[30]

K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In CVPR, pages 770–778, 2016.

[31]

D. P. Kingma and J. Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.

[32]

B. Settles and M. Craven. An analysis of active learning strategies for sequence labeling tasks. In EMNLP, pages 1070–1079, 2008.

Cited By

Mazumder SBanipal IAsthana SZhang B(2024)Label Engineering Methods for ML SystemsIntelligent Systems and Applications10.1007/978-3-031-66336-9_33(464-474)Online publication date: 1-Aug-2024
https://doi.org/10.1007/978-3-031-66336-9_33
Liu SChen KHu TMao Y(2023)Uncertainty-aware complementary label queries for active learning基于主动学习的不确定性感知补标签查询Frontiers of Information Technology & Electronic Engineering10.1631/FITEE.220058924:10(1497-1503)Online publication date: 7-Nov-2023
https://doi.org/10.1631/FITEE.2200589

Index Terms

Complementary Label Queries for Efficient Active Learning
1. Computing methodologies
  1. Machine learning

Recommendations

Incremental Multi-Label Learning with Active Queries
Abstract
In multi-label learning, it is rather expensive to label instances since they are simultaneously associated with multiple labels. Therefore, active learning, which reduces the labeling cost by actively querying the labels of the most valuable data,...
Graph-Based Active Learning Based on Label Propagation
MDAI '08 Sabadell: Proceedings of the 5th International Conference on Modeling Decisions for Artificial Intelligence

By only selecting the most informative instances for labeling, active learning could reduce the labeling cost when labeled instances are hard to obtain. Facing the same situation, semi-supervised learning utilize unlabeled instances to strengthen ...
Improved prototypical network for active few-shot learning
Highlights
- The novel active few-shot learning task is proposed to better accomplish classification with few labeled examples.
- AC-FSL model is designed by introducing the active loss prediction module into the few-shot prototype network.
- ...
Abstract
Different from deep learning with large scale supervision, few-shot learning aims to learn the samples’ characteristics from few labeled examples. Apparently, few-shot learning is more in line with the visual cognitive mechanism of the human ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

ICIGP '23: Proceedings of the 2023 6th International Conference on Image and Graphics Processing

January 2023

246 pages

ISBN:9781450398572

DOI:10.1145/3582649

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 April 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

ICIGP 2023

ICIGP 2023: 2023 The 6th International Conference on Image and Graphics Processing

January 6 - 8, 2023

Chongqing, China

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
78
Total Downloads

Downloads (Last 12 months)53
Downloads (Last 6 weeks)4

Reflects downloads up to 23 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Mazumder SBanipal IAsthana SZhang B(2024)Label Engineering Methods for ML SystemsIntelligent Systems and Applications10.1007/978-3-031-66336-9_33(464-474)Online publication date: 1-Aug-2024
https://doi.org/10.1007/978-3-031-66336-9_33
Liu SChen KHu TMao Y(2023)Uncertainty-aware complementary label queries for active learning基于主动学习的不确定性感知补标签查询Frontiers of Information Technology & Electronic Engineering10.1631/FITEE.220058924:10(1497-1503)Online publication date: 7-Nov-2023
https://doi.org/10.1631/FITEE.2200589

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents