research-article

Cycle-consistent Conditional Adversarial Transfer Networks

Authors:

Zhengming Ding,

Zi HuangAuthors Info & Claims

MM '19: Proceedings of the 27th ACM International Conference on Multimedia

Pages 747 - 755

https://doi.org/10.1145/3343031.3350902

Published: 15 October 2019 Publication History

Abstract

Domain adaptation investigates the problem of cross-domain knowledge transfer where the labeled source domain and unlabeled target domain have distinctive data distributions. Recently, adversarial training have been successfully applied to domain adaptation and achieved state-of-the-art performance. However, there is still a fatal weakness existing in current adversarial models which is raised from the equilibrium challenge of adversarial training. Specifically, although most of existing methods are able to confuse the domain discriminator, they cannot guarantee that the source domain and target domain are sufficiently similar. In this paper, we propose a novel approach named cycle-consistent conditional adversarial transfer networks (3CATN) to handle this issue. Our approach takes care of the domain alignment by leveraging adversarial training. Specifically, we condition the adversarial networks with the cross-covariance of learned features and classifier predictions to capture the multimodal structures of data distributions. However, since the classifier predictions are not certainty information, a strong condition with the predictions is risky when the predictions are not accurate. We, therefore, further propose that the truly domain-invariant features should be able to be translated from one domain to the other. To this end, we introduce two feature translation losses and one cycle-consistent loss into the conditional adversarial domain adaptation networks. Extensive experiments on both classical and large-scale datasets verify that our model is able to outperform previous state-of-the-arts with significant improvements.

References

[1]

Rahaf Aljundi, Rémi Emonet, Damien Muselet, and Marc Sebban. 2015. Landmarks-based kernelized subspace alignment for unsupervised domain adaptation. In CVPR. 56--63.

[2]

Sanjeev Arora, Rong Ge, Yingyu Liang, Tengyu Ma, and Yi Zhang. 2017. Generalization and equilibrium in generative adversarial nets (gans). In ICML. JMLR. org, 224--232.

[3]

Konstantinos Bousmalis, Nathan Silberman, David Dohan, Dumitru Erhan, and Dilip Krishnan. 2017. Unsupervised pixel-level domain adaptation with generative adversarial networks. In CVPR, Vol. 1. 7--16.

[4]

Yi-Hsin Chen, Wei-Yu Chen, Yu-Ting Chen, Bo-Cheng Tsai, Yu-Chiang Frank Wang, and Min Sun. 2017. No more discrimination: Cross city adaptation of road scene segmenters. In CVPR. 1992--2001.

[5]

Zhengming Ding, Nasser M Nasrabadi, and Yun Fu. 2018. Semi-supervised Deep Domain Adaptation via Coupled Neural Networks. IEEE TIP 27, 11 (2018), 5214--5224.

[6]

Zhengming Ding, Ming Shao, and Yun Fu. 2014. Latent Low-Rank Transfer Subspace Learning for Missing Modality Recognition. In AAAI. 1192--1198.

[7]

Yaroslav Ganin and Victor Lempitsky. 2014. Unsupervised domain adaptation by backpropagation. arXiv preprint arXiv:1409.7495 (2014).

Digital Library

[8]

Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario Marchand, and Victor Lempitsky. 2016. Domain-adversarial training of neural networks. JMLR 17, 1 (2016), 2096--2030.

Digital Library

[9]

Muhammad Ghifary, W Bastiaan Kleijn, Mengjie Zhang, David Balduzzi, and Wen Li. 2016. Deep reconstruction-classification networks for unsupervised domain adaptation. In ECCV. Springer, 597--613.

[10]

Boqing Gong, Yuan Shi, Fei Sha, and Kristen Grauman. 2012. Geodesic flow kernel for unsupervised domain adaptation. In CVPR. IEEE, 2066--2073.

[11]

Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative adversarial nets. In NIPS. 2672--2680.

[12]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In CVPR. 770--778.

[13]

Judy Hoffman, Eric Tzeng, Taesung Park, Jun-Yan Zhu, Phillip Isola, Kate Saenko, Alexei Efros, and Trevor Darrell. 2018. CyCADA: Cycle-Consistent Adversarial Domain Adaptation. In ICML. 1994--2003.

[14]

Weixiang Hong, Zhenzhen Wang, Ming Yang, and Junsong Yuan. 2018. Conditional generative adversarial network for structured domain adaptation. In CVPR. 1335--1344.

[15]

Yao-Hung Hubert Tsai, Yi-Ren Yeh, and Yu-Chiang Frank Wang. 2016. Learning cross-domain landmarks for heterogeneous domain adaptation. In CVPR. 5081-- 5090.

[16]

Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. 2012. Imagenet classification with deep convolutional neural networks. In NIPS. 1097--1105.

[17]

Jingjing Li, Mengmeng Jing, Ke Lu, Zhengming Ding, Lei Zhu, and Zi Huang. 2019. Leveraging the Invariant Side of Generative Zero-Shot Learning. In CVPR. 7402--7411.

[18]

Jingjing Li, Mengmeng Jing, Ke Lu, Lei Zhu, and Heng Tao Shen. 2019. Locality Preserving Joint Transfer for Domain Adaptation. IEEE TIP (2019).

Digital Library

[19]

Jingjing Li, Mengmeng Jing, Ke Lu, Lei Zhu, Yang Yang, and Zi Huang. 2019. From Zero-Shot Learning to Cold-Start Recommendation. AAAI (2019).

[20]

Jingjing Li, Ke Lu, Zi Huang, and Heng Tao Shen. 2017. Two birds one stone: on both cold-start and long-tail recommendation. In ACM MM. ACM, 898--906.

[21]

Jingjing Li, Ke Lu, Zi Huang, Lei Zhu, and Heng Tao Shen. 2018. Transfer Independently Together: A Generalized Framework for Domain Adaptation. IEEE TCYB (2018).

[22]

Jingjing Li, Jidong Zhao, and Ke Lu. 2016. Joint Feature Selection and Structure Preservation for Domain Adaptation. In IJCAI. 1697--1703.

[23]

Ming-Yu Liu, Thomas Breuel, and Jan Kautz. 2017. Unsupervised image-to-image translation networks. In NIPS. 700--708.

[24]

Ming-Yu Liu and Oncel Tuzel. 2016. Coupled generative adversarial networks. In NIPS. 469--477.

[25]

Mingsheng Long, Yue Cao, Jianmin Wang, and Michael I Jordan. 2015. Learning transferable features with deep adaptation networks. In ICML. 97--105.

[26]

Mingsheng Long, Zhangjie Cao, Jianmin Wang, and Michael I Jordan. 2018. Conditional adversarial domain adaptation. In NIPS. 1647--1657.

[27]

Mingsheng Long, Han Zhu, Jianmin Wang, and Michael I Jordan. 2016. Unsupervised domain adaptation with residual transfer networks. In NIPS. 136--144.

[28]

Mingsheng Long, Han Zhu, Jianmin Wang, and Michael I Jordan. 2017. Deep Transfer Learning with Joint Adaptation Networks. In ICML. 2208--2217.

[29]

Mehdi Mirza and Simon Osindero. 2014. Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784 (2014).

[30]

Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng. 2011. Reading digits in natural images with unsupervised feature learning. In NIPS workshop on deep learning and unsupervised feature learning, Vol. 2011. 5.

[31]

Sinno Jialin Pan, Ivor W Tsang, James T Kwok, and Qiang Yang. 2011. Domain adaptation via transfer component analysis. IEEE TNN 22, 2 (2011), 199--210.

[32]

Sinno Jialin Pan, Qiang Yang, et al. 2010. A survey on transfer learning. IEEE TKDE 22, 10 (2010), 1345--1359.

[33]

Xingchao Peng, Ben Usman, Neela Kaushik, Judy Hoffman, Dequan Wang, and Kate Saenko. 2017. VisDA: The Visual Domain Adaptation Challenge. arXiv:arXiv:1710.06924

[34]

Pedro O Pinheiro and AI Element. 2018. Unsupervised domain adaptation with similarity learning. In CVPR. 8004--8013.

[35]

Kate Saenko, Brian Kulis, Mario Fritz, and Trevor Darrell. 2010. Adapting visual category models to new domains. In European conference on computer vision. Springer, 213--226.

Digital Library

[36]

Kuniaki Saito, Kohei Watanabe, Yoshitaka Ushiku, and Tatsuya Harada. 2018. Maximum classifier discrepancy for unsupervised domain adaptation. CVPR (2018).

[37]

Swami Sankaranarayanan, Yogesh Balaji, Carlos D Castillo, and Rama Chellappa. 2018. Generate to adapt: Aligning domains using generative adversarial networks. In CVPR 2018. 8503--8512.

[38]

Le Song, Jonathan Huang, Alex Smola, and Kenji Fukumizu. 2009. Hilbert space embeddings of conditional distributions with applications to dynamical systems. In ICML. ACM, 961--968.

[39]

Masashi Sugiyama, Matthias Krauledat, and Klaus-Robert MÃller. 2007. Covariate shift adaptation by importance weighted cross validation. JMLR 8, May (2007), 985--1005.

[40]

Baochen Sun and Kate Saenko. 2016. Deep coral: Correlation alignment for deep domain adaptation. In ECCV. Springer, 443--450.

[41]

Yaniv Taigman, Adam Polyak, and Lior Wolf. 2016. Unsupervised cross-domain image generation. arXiv preprint arXiv:1611.02200 (2016).

[42]

Eric Tzeng, Judy Hoffman, Kate Saenko, and Trevor Darrell. 2017. Adversarial discriminative domain adaptation. In CVPR, Vol. 1. 4--13.

[43]

Eric Tzeng, Judy Hoffman, Ning Zhang, Kate Saenko, and Trevor Darrell. 2014. Deep domain confusion: Maximizing for domain invariance. arXiv preprint arXiv:1412.3474 (2014).

[44]

Yongqin Xian, Bernt Schiele, and Zeynep Akata. 2017. Zero-Shot Learning-The Good, the Bad and the Ugly. arXiv preprint arXiv:1703.04394 (2017).

[45]

Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. 2017. Unpaired image-to-image translation using cycle-consistent adversarial networks. In ICCV. 2223--2232.

Cited By

Gong KGao YDong W(2024)Privacy-Preserving and Cross-Domain Human Sensing by Federated Domain Adaptation with Semantic Knowledge CorrectionProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36435038:1(1-26)Online publication date: 6-Mar-2024
https://dl.acm.org/doi/10.1145/3643503
Wang MChen JWang YWang Sli LSu HGong ZWu KChen Z(2024)Joint Adversarial Domain Adaptation With Structural Graph AlignmentIEEE Transactions on Network Science and Engineering10.1109/TNSE.2023.330257411:1(604-612)Online publication date: Jan-2024
https://doi.org/10.1109/TNSE.2023.3302574
Ru JTian JXiao CLi JShen H(2024)Imbalanced Open Set Domain Adaptation via Moving-Threshold Estimation and Gradual AlignmentIEEE Transactions on Multimedia10.1109/TMM.2023.329776826(2504-2514)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TMM.2023.3297768
Show More Cited By

Index Terms

Cycle-consistent Conditional Adversarial Transfer Networks
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
  2. Machine learning
    1. Learning paradigms
      1. Multi-task learning
        Transfer learning
    2. Machine learning approaches
      1. Neural networks

Recommendations

Pairwise Similarity Regularization for Adversarial Domain Adaptation
MM '20: Proceedings of the 28th ACM International Conference on Multimedia

Domain adaptation aims at learning a predictive model that can generalize to a new target domain different from the source (training) domain. To mitigate the domain gap, adversarial training has been developed to learn domain invariant representations. ...
Unsupervised domain adaptation with adversarial distribution adaptation network
Abstract
Adversarial domain adaptation is a powerful approach to transfer the knowledge of the label-rich source domain to the label-scarce target domain by mitigating domain shifts across distributions. Existing domain adaptation methods align either the ...
Domain Adaptation for Speaker Verification Based on Self-supervised Learning with Adversarial Training
MultiMedia Modeling
Abstract
Speaker verification models trained on a single domain have difficulty keeping performance on new domain data. Adversarial training maps different domain data to the same subspace to handle this problem. However, adversarial training only uses ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

MM '19: Proceedings of the 27th ACM International Conference on Multimedia

October 2019

2794 pages

ISBN:9781450368896

DOI:10.1145/3343031

General Chairs:
Laurent Amsaleg
CNRS-IRISA, France
,
Benoit Huet
EURECOM, France
,
Martha Larson
Radboud University and TU Delft (Netherlands)
,
Program Chairs:
Guillaume Gravier
CNRS-IRISA, France
,
Hayley Hung
Delft University of Technology Netherlands
,
Chong-Wah Ngo
City University of Hong Kong Hong Kong
,
Wei Tsang Ooi
National University of Singapore Singapore

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 15 October 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

NSFC

Conference

MM '19

Sponsor:

SIGMM

MM '19: The 27th ACM International Conference on Multimedia

October 21 - 25, 2019

Nice, France

Acceptance Rates

MM '19 Paper Acceptance Rate 252 of 936 submissions, 27%;

Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

59
Total Citations
View Citations
702
Total Downloads

Downloads (Last 12 months)28
Downloads (Last 6 weeks)0

Reflects downloads up to 08 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Gong KGao YDong W(2024)Privacy-Preserving and Cross-Domain Human Sensing by Federated Domain Adaptation with Semantic Knowledge CorrectionProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36435038:1(1-26)Online publication date: 6-Mar-2024
https://dl.acm.org/doi/10.1145/3643503
Wang MChen JWang YWang Sli LSu HGong ZWu KChen Z(2024)Joint Adversarial Domain Adaptation With Structural Graph AlignmentIEEE Transactions on Network Science and Engineering10.1109/TNSE.2023.330257411:1(604-612)Online publication date: Jan-2024
https://doi.org/10.1109/TNSE.2023.3302574
Ru JTian JXiao CLi JShen H(2024)Imbalanced Open Set Domain Adaptation via Moving-Threshold Estimation and Gradual AlignmentIEEE Transactions on Multimedia10.1109/TMM.2023.329776826(2504-2514)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TMM.2023.3297768
Zhou YLi XWu Y(2024)Cycle-Consistent Adversarial chest X-rays Domain Adaptation for pneumonia diagnosisNeurocomputing10.1016/j.neucom.2024.128604610(128604)Online publication date: Dec-2024
https://doi.org/10.1016/j.neucom.2024.128604
Li JZhu LDu ZLi JZhu LDu Z(2024)Continual Test-Time Unsupervised Domain AdaptationUnsupervised Domain Adaptation10.1007/978-981-97-1025-6_7(191-212)Online publication date: 16-Feb-2024
https://doi.org/10.1007/978-981-97-1025-6_7
Li JZhu LDu ZLi JZhu LDu Z(2024)Bi-Classifier Adversarial Learning-Based Unsupervised Domain AdaptationUnsupervised Domain Adaptation10.1007/978-981-97-1025-6_4(69-104)Online publication date: 16-Feb-2024
https://doi.org/10.1007/978-981-97-1025-6_4
Li JZhu LDu ZLi JZhu LDu Z(2024)Criterion Optimization-Based Unsupervised Domain AdaptationUnsupervised Domain Adaptation10.1007/978-981-97-1025-6_3(19-67)Online publication date: 16-Feb-2024
https://doi.org/10.1007/978-981-97-1025-6_3
Su HLi JDu ZZhu LLu KShen H(2023)Cross-domain Recommendation via Dual Adversarial AdaptationACM Transactions on Information Systems10.1145/363252442:3(1-26)Online publication date: 11-Nov-2023
https://dl.acm.org/doi/10.1145/3632524
Yu ZLi JDu ZLi FZhu LYang YEl Saddik AMei TCucchiara RBertini MTobon Vallejo DAtrey PHossain M(2023)Noise-Robust Continual Test-Time Domain AdaptationProceedings of the 31st ACM International Conference on Multimedia10.1145/3581783.3612071(2654-2662)Online publication date: 26-Oct-2023
https://dl.acm.org/doi/10.1145/3581783.3612071
Wang LZhang HXiao QXu HShen CJin X(2023)Effective Eyebrow Matting with Domain AdaptationComputer Graphics Forum10.1111/cgf.1468241:7(347-358)Online publication date: 20-Mar-2023
https://doi.org/10.1111/cgf.14682
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten