research-article

Universal Domain Adaptive Object Detector

Authors:

Shiliang PuAuthors Info & Claims

MM '22: Proceedings of the 30th ACM International Conference on Multimedia

Pages 2258 - 2266

https://doi.org/10.1145/3503161.3547937

Published: 10 October 2022 Publication History

Abstract

Universal domain adaptive object detection (UniDAOD) is more challenging than domain adaptive object detection (DAOD) since the label space of the source domain may not be the same as that of the target and the scale of objects in the universal scenarios can vary dramatically (i.e, category shift and scale shift). To this end, we propose US-DAF, namely Universal Scale-Aware Domain Adaptive Faster RCNN with Multi-Label Learning, to reduce the negative transfer effect during training while maximizing transferability as well as discriminability in both domains under a variety of scales. Specifically, our method is implemented by two modules: 1) We facilitate the feature alignment of common classes and suppress the interference of private classes by designing a Filter Mechanism module to overcome the negative transfer caused by category shift. 2) We fill the blank of scale-aware adaptation in object detection by introducing a new Multi-Label Scale-Aware Adapter to perform individual alignment between corresponding scale for two domains. Experiments show that US-DAF achieves state-of-the-art results on three scenarios (\emphi.e, Open-Set, Partial-Set, and Closed-Set) and yields 7.1% and 5.9% relative improvement on benchmark datasets Clipart1k and Watercolor in particular.

Supplementary Material

MP4 File (MM22-fp775.mp4)

In this paper, we introduce a novel setting that better meets the needs of real-world scenarios, Universal Domain Adaptive Object Detection (UniDAOD), which requires no prior knowledge on the label set of target domains. In order to meet this challenge of UniDAOD, we contribute a Universal Scale-Aware Domain Adaptive Faster R-CNN with Multi-Label Learning (US-DAF) framework, which, to the best of our knowledge, is a pioneer work for object detection under both category shift and scale issue toward universal scenarios.

Download
17.19 MB

References

[1]

Zhangjie Cao, Lijia Ma, Mingsheng Long, and Jianmin Wang. 2018. Partial adversarial domain adaptation. In ECCV. 135--150.

[2]

Chaoqi Chen, Zebiao Zheng, Xinghao Ding, Yue Huang, and Qi Dou. 2020. Harmonizing transferability and discriminability for adapting object detectors. In CVPR. 8869--8878.

[3]

Yuhua Chen, Dengxin Dai, Jordi Pont-Tuset, and Luc Van Gool. 2016. Scale-aware alignment of hierarchical image segmentation. In CVPR. 364--372.

[4]

Yuhua Chen, Wen Li, Christos Sakaridis, Dengxin Dai, and Luc Van Gool. 2018. Domain adaptive faster r-cnn for object detection in the wild. In CVPR. 3339--3348.

[5]

Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele. 2016. The cityscapes dataset for semantic urban scene understanding. In CVPR. 3213--3223.

[6]

Mark Everingham, SM Ali Eslami, Luc Van Gool, Christopher KI Williams, John Winn, and Andrew Zisserman. 2015. The pascal visual object classes challenge: A retrospective. International journal of computer vision 111, 1 (2015), 98--136.

[7]

Bo Fu, Zhangjie Cao, Mingsheng Long, and Jianmin Wang. 2020. Learning to detect open classes for universal domain adaptation. In ECCV. 567--583.

[8]

Cheng-Yang Fu, Wei Liu, Ananth Ranga, Ambrish Tyagi, and Alexander C Berg. 2017. Dssd: Deconvolutional single shot detector. arXiv:1701.06659

[9]

Yaroslav Ganin and Victor Lempitsky. 2015. Unsupervised domain adaptation by backpropagation. In International conference on machine learning. 1180--1189.

Digital Library

[10]

Ross Girshick. 2015. Fast r-cnn. In ICCV. 1440--1448.

[11]

Yunchao Gong, Yangqing Jia, Thomas Leung, Alexander Toshev, and Sergey Ioffe. 2013. Deep convolutional ranking for multilabel image annotation. arXiv:1312.4894

[12]

Yves Grandvalet and Yoshua Bengio. 2004. Semi-supervised learning by entropy minimization. Advances in neural information processing systems 17 (2004).

[13]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In CVPR. 770--778.

[14]

Zhenwei He and Lei Zhang. 2019. Multi-adversarial faster-rcnn for unrestricted object detection. In ICCV. 6668--6677.

[15]

Zhenwei He and Lei Zhang. 2020. Domain adaptive object detection via asymmetric tri-way faster-rcnn. In ECCV. 309--324.

[16]

Naoto Inoue, Ryosuke Furuta, Toshihiko Yamasaki, and Kiyoharu Aizawa. 2018. Cross-domain weakly-supervised object detection through progressive domain adaptation. In CVPR. 5001--5009.

[17]

Mehran Khodabandeh, Arash Vahdat, Mani Ranjbar, and William G Macready. 2019. A robust learning approach to domain adaptive object detection. In ICCV. 480--490.

[18]

Taekyung Kim, Minki Jeong, Seunghyeon Kim, Seokeon Choi, and Changick Kim. 2019. Diversify and match: A domain adaptive representation learning paradigm for object detection. In CVPR. 12456--12465.

[19]

Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. 2012. Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems 25 (2012), 1097--1105.

[20]

Qing Lian, Fengmao Lv, Lixin Duan, and Boqing Gong. 2019. Constructing self-motivated pyramid curriculums for cross-domain semantic segmentation: A non-adversarial approach. In ICCV. 6758--6767.

[21]

Tsung-Yi Lin, Piotr Dollár, Ross Girshick, Kaiming He, Bharath Hariharan, and Serge Belongie. 2017. Feature pyramid networks for object detection. In CVPR. 2117--2125.

[22]

Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott Reed, Cheng-Yang Fu, and Alexander C Berg. 2016. Ssd: Single shot multibox detector. In ECCV. 21--37.

[23]

Mingsheng Long, Yue Cao, Jianmin Wang, and Michael Jordan. 2015. Learning transferable features with deep adaptation networks. In ICML. 97--105.

[24]

Sinno Jialin Pan and Qiang Yang. 2009. A survey on transfer learning. IEEE Transactions on knowledge and data engineering 22, 10 (2009), 1345--1359.

Digital Library

[25]

Pau Panareda Busto and Juergen Gall. 2017. Open set domain adaptation. In ICCV. 754--763.

[26]

Joseph Redmon, Santosh Divvala, Ross Girshick, and Ali Farhadi. 2016. You only look once: Unified, real-time object detection. In CVPR. 779--788.

[27]

Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. 2015. Faster r-cnn: Towards real-time object detection with region proposal networks. In NeuIPS. 91--99.

[28]

Kuniaki Saito, Yoshitaka Ushiku, Tatsuya Harada, and Kate Saenko. 2019. Strong-weak distribution alignment for adaptive object detection. In CVPR. 6956--6965.

[29]

Christos Sakaridis, Dengxin Dai, and Luc Van Gool. 2018. Semantic foggy scene understanding with synthetic data. International Journal of Computer Vision 126, 9 (2018), 973--992.

Digital Library

[30]

Zhiqiang Shen, Harsh Maheshwari, Weichen Yao, and Marios Savvides. 2019. Scl: Towards accurate domain adaptive object detection via gradient detach based stacked complementary losses. arXiv:1911.02559

[31]

Laurens Van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE. Journal of machine learning research 9, 11 (2008).

[32]

Rongchang Xie, Fei Yu, Jiachao Wang, Yizhou Wang, and Li Zhang. 2019. Multi-level Domain Adaptive learning for Cross-Domain Detection. arXiv:1907.11484

[33]

Chang-Dong Xu, Xing-Ran Zhao, Xin Jin, and Xiu-Shen Wei. 2020. Exploring categorical regularization for domain adaptive object detection. In CVPR. 11724--11733.

[34]

Kaichao You, Mingsheng Long, Zhangjie Cao, Jianmin Wang, and Michael I Jordan. 2019. Universal domain adaptation. In CVPR. 2720--2729.

[35]

Min-Ling Zhang and Zhi-Hua Zhou. 2006. Multilabel neural networks with applications to functional genomics and text categorization. IEEE transactions on Knowledge and Data Engineering 18, 10 (2006), 1338--1351.

Digital Library

[36]

Weichen Zhang, Wanli Ouyang, Wen Li, and Dong Xu. 2018. Collaborative and adversarial network for unsupervised domain adaptation. In CVPR. 3801--3809.

[37]

Yang Zhang, Philip David, and Boqing Gong. 2017. Curriculum domain adaptation for semantic segmentation of urban scenes. In ICCV. 2020--2030.

[38]

Zhen Zhao, Yuhong Guo, Haifeng Shen, and Jieping Ye. 2020. Adaptive object detection with dual multi-label prediction. In ECCV. 54--69.

[39]

Yangtao Zheng, Di Huang, Songtao Liu, and Yunhong Wang. 2020. Cross-domain object detection through coarse-to-fine feature adaptation. In CVPR. 13766--13775.

[40]

Xinge Zhu, Jiangmiao Pang, Ceyuan Yang, Jianping Shi, and Dahua Lin. 2019. Adapting object detectors via selective cross-domain alignment. In CVPR. 687--696.

Cited By

Zhou YZhang YZhang LHua ZCai JKankanhalli MPrabhakaran BBoll SSubramanian RZheng LSingh VCesar PXie LXu D(2024)DERD: Data-free Adversarial Robustness Distillation through Self-adversarial Teacher GroupProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3680796(10055-10064)Online publication date: 28-Oct-2024
https://dl.acm.org/doi/10.1145/3664647.3680796
Yang XWang HSun JXiao YXiang WChen CHua XLuo X(2024)ROSE: Relational and Prototypical Structure Learning for Universal Domain Adaptive HashingIEEE Transactions on Information Forensics and Security10.1109/TIFS.2024.344431919(7690-7704)Online publication date: 2024
https://doi.org/10.1109/TIFS.2024.3444319
Tu HGuo YWang WJin RChen SWu Z(2024)Domain Adaptive SAR Ship Detection based on Knowledge Distillation and Transformer Attention2024 IEEE International Conference on Signal, Information and Data Processing (ICSIDP)10.1109/ICSIDP62679.2024.10867854(1-5)Online publication date: 22-Nov-2024
https://doi.org/10.1109/ICSIDP62679.2024.10867854
Show More Cited By

Index Terms

Universal Domain Adaptive Object Detector
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
        Object detection

Recommendations

Domain consensual contrastive learning for few-shot universal domain adaptation
Abstract
Traditional unsupervised domain adaptation (UDA) aims to transfer the learned knowledge from a fully labeled source domain to another unlabeled target domain on the same label set. The strong assumptions of full annotations on the source domain ...
Source data‐free domain adaptation of object detector through domain‐specific perturbation
Abstract
The current unsupervised cross‐domain detection methods need source domain data to retrain the detection model in target domain. However, the source domain data may be unavailable due to privacy, decentralization, or computation resource ...
Cross Classroom Domain Adaptive Object Detector for Student’s Heads
Artificial Neural Networks and Machine Learning – ICANN 2023
Abstract
Training on a label-rich dataset and test on another label-scarce dataset usually leads to a poor performance because of the domain shift. Unsupervised domain adaptation is proved to be effective on this problem in recent researches. Unsupervised ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

MM '22: Proceedings of the 30th ACM International Conference on Multimedia

October 2022

7537 pages

ISBN:9781450392037

DOI:10.1145/3503161

General Chairs:
João Magalhães
NOVA University of Lisbon, Portugal
,
Alberto del Bimbo
University of Florence, Italy
,
Shin'ichi Satoh
National Institute of Informatics, Japan
,
Nicu Sebe
University of Trento, Italy
,
Program Chairs:
Xavier Alameda-Pineda
Inria, Grenoble, France
,
Qin Jin
Renmin University of China, China
,
Vincent Oria
New Jersey Institute of Technology, USA
,
Laura Toni
University College London, UK

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 10 October 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National Key R&D Program of China
Chongqing Natural Science Fund
CCF Hikvision Open Fund
CAAI-Huawei MindSpore Open Fund
Beijing Academy of Artificial Intelligence (BAAI)

Conference

MM '22

Sponsor:

SIGMM

MM '22: The 30th ACM International Conference on Multimedia

October 10 - 14, 2022

Lisboa, Portugal

Acceptance Rates

Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

16
Total Citations
View Citations
159
Total Downloads

Downloads (Last 12 months)33
Downloads (Last 6 weeks)7

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Zhou YZhang YZhang LHua ZCai JKankanhalli MPrabhakaran BBoll SSubramanian RZheng LSingh VCesar PXie LXu D(2024)DERD: Data-free Adversarial Robustness Distillation through Self-adversarial Teacher GroupProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3680796(10055-10064)Online publication date: 28-Oct-2024
https://dl.acm.org/doi/10.1145/3664647.3680796
Yang XWang HSun JXiao YXiang WChen CHua XLuo X(2024)ROSE: Relational and Prototypical Structure Learning for Universal Domain Adaptive HashingIEEE Transactions on Information Forensics and Security10.1109/TIFS.2024.344431919(7690-7704)Online publication date: 2024
https://doi.org/10.1109/TIFS.2024.3444319
Tu HGuo YWang WJin RChen SWu Z(2024)Domain Adaptive SAR Ship Detection based on Knowledge Distillation and Transformer Attention2024 IEEE International Conference on Signal, Information and Data Processing (ICSIDP)10.1109/ICSIDP62679.2024.10867854(1-5)Online publication date: 22-Nov-2024
https://doi.org/10.1109/ICSIDP62679.2024.10867854
Lin LLiu QZheng XLin Z(2024)Slow-Fast Adaptation for Source-Free Object Detection2024 IEEE International Conference on Multimedia and Expo (ICME)10.1109/ICME57554.2024.10688042(1-6)Online publication date: 15-Jul-2024
https://doi.org/10.1109/ICME57554.2024.10688042
Danish MIqbal JAli MSarfraz MKhan SKhan M(2024)Perturbing Dominant Feature Modes for Single Domain-Generalized Object Detection2024 International Conference on Digital Image Computing: Techniques and Applications (DICTA)10.1109/DICTA63115.2024.00026(93-100)Online publication date: 27-Nov-2024
https://doi.org/10.1109/DICTA63115.2024.00026
Shi WLiu DWu ZZheng B(2024)Confused and disentangled distribution alignment for unsupervised universal adaptive object detectionKnowledge-Based Systems10.1016/j.knosys.2024.112085300(112085)Online publication date: Sep-2024
https://doi.org/10.1016/j.knosys.2024.112085
Shi WLiu DTan DZheng B(2024)A dynamically class-wise weighting mechanism for unsupervised cross-domain object detection under universal scenariosKnowledge-Based Systems10.1016/j.knosys.2024.111987299(111987)Online publication date: Sep-2024
https://doi.org/10.1016/j.knosys.2024.111987
Gao FPi DChen J(2024)Balanced and robust unsupervised Open Set Domain Adaptation via joint adversarial alignment and unknown class isolationExpert Systems with Applications10.1016/j.eswa.2023.122127238(122127)Online publication date: Mar-2024
https://doi.org/10.1016/j.eswa.2023.122127
Zhang LQin LXu MChen WPu SZhang W(2023)Randomized Spectrum Transformations for Adapting Object Detector in Unseen DomainsIEEE Transactions on Image Processing10.1109/TIP.2023.330691532(4868-4879)Online publication date: 2023
https://doi.org/10.1109/TIP.2023.3306915
Jiang GZhu PWang YHu Q(2023)OpenMix+: Revisiting Data Augmentation for Open Set RecognitionIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2023.326868033:11(6777-6787)Online publication date: 20-Apr-2023
https://dl.acm.org/doi/10.1109/TCSVT.2023.3268680
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten