Article

Return of frustratingly easy domain adaptation

Authors:

Kate SaenkoAuthors Info & Claims

AAAI'16: Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence

Pages 2058 - 2065

Published: 12 February 2016 Publication History

Abstract

Unlike human learning, machine learning often fails to handle changes between training (source) and test (target) input distributions. Such domain shifts, common in practical scenarios, severely damage the performance of conventional machine learning methods. Supervised domain adaptation methods have been proposed for the case when the target data have labels, including some that perform very well despite being "frustratingly easy" to implement. However, in practice, the target domain is often unlabeled, requiring unsupervised adaptation. We propose a simple, effective, and efficient method for unsupervised domain adaptation called CORrelation ALignment (CORAL). CORAL minimizes domain shift by aligning the second-order statistics of source and target distributions, without requiring any target labels. Even though it is extraordinarily simple–it can be implemented in four lines of Matlab code–CORAL performs remarkably well in extensive evaluations on standard benchmark datasets.

References

[1]

Ben-David, S.; Blitzer, J.; Crammer, K.; and Pereira, F. 2007. Analysis of representations for domain adaptation. In NIPS.

Digital Library

[2]

Blitzer, J.; Dredze, M.; and Pereira, F. 2007. Biographies, Bollywood, Boom-boxes and Blenders: Domain Adaptation for Sentiment Classification. In ACL.

[3]

Blitzer, J.; McDonald, R.; and Pereira, F. 2006. Domain adaptation with structural correspondence learning. In EMNLP.

Digital Library

[4]

Borgwardt, K. M.; Gretton, A.; Rasch, M. J.; Kriegel, H.-P.; Schölkopf, B.; and Smola, A. J. 2006. Integrating structured biological data by kernel maximum mean discrepancy. In Bioinformatics.

Digital Library

[5]

Cai, J.-F.; Candès, E. J.; and Shen, Z. 2010. A singular value thresholding algorithm for matrix completion. SIAM J. on Optimization 20(4):1956-1982.

Digital Library

[6]

Caseiro, R.; Henriques, J. F.; Martins, P.; and Batista, J. 2015. Beyond the shortest path : Unsupervised domain adaptation by sampling subspaces along the spline flow. In CVPR.

[7]

Chopra, S.; Balakrishnan, S.; and Gopalan, R. 2013. Dlid: Deep learning for domain adaptation by interpolating between domains. In ICML Workshop.

[8]

Daume III, H. 2007. Frustratingly easy domain adaptation. In ACL.

[9]

Deng, J.; Dong, W.; Socher, R.; Li, L.-J.; Li, K.; and Fei-Fei, L. 2009. Imagenet: A large-scale hierarchical image database. In CVPR.

[10]

Donahue, J.; Jia, Y.; Vinyals, O.; Hoffman, J.; Zhang, N.; Tzeng, E.; and Darrell, T. 2014. Decaf: A deep convolutional activation feature for generic visual recognition. In ICML.

Digital Library

[11]

Duan, L.; Tsang, I. W.; Xu, D.; and Chua, T. 2009. Domain adaptation from multiple sources via auxiliary classifiers. In ICML.

Digital Library

[12]

Duan, L.; Tsang, I. W.; and Xu, D. 2012. Domain transfer multiple kernel learning. TPAMI 34(3):465-479.

Digital Library

[13]

Fernando, B.; Habrard, A.; Sebban, M.; and Tuytelaars, T. 2013. Unsupervised visual domain adaptation using subspace alignment. In ICCV.

Digital Library

[14]

Ganin, Y., and Lempitsky, V. 2015. Unsupervised domain adaptation by backpropagation. In ICML.

[15]

Ghifary, M.; Kleijn, W. B.; and Zhang, M. 2014. Domain adaptive neural networks for object recognition. In PRICAI.

[16]

Gong, B.; Shi, Y.; Sha, F.; and Grauman, K. 2012. Geodesic flow kernel for unsupervised domain adaptation. In CVPR.

[17]

Gong, B.; Grauman, K.; and Sha, F. 2013. Connecting the dots with landmarks: Discriminatively learning domain-invariant features for unsupervised domain adaptation. In ICML.

[18]

Gopalan, R.; Li, R.; and Chellappa, R. 2011. Domain adaptation for object recognition: An unsupervised approach. In ICCV.

Digital Library

[19]

Gregory, G.; Alex, H.; and Pietro, P. 2007. Caltech 256 object category dataset. In Tech. Rep. UCB/CSD-04-1366, California Institue of Technology.

[20]

Harel, M., and Mannor, S. 2011. Learning from multiple outlooks. In ICML.

[21]

Hariharan, B.; Malik, J.; and Ramanan, D. 2012. Discriminative decorrelation for clustering and classification. In ECCV.

Digital Library

[22]

Huang, D.-A., and Wang, Y.-C. 2013. Coupled dictionary and feature space learning with applications to cross-domain image synthesis and recognition. In ICCV.

Digital Library

[23]

Huang, J.; Smola, A. J.; Gretton, A.; Borgwardt, K. M.; and Schölkopf, B. 2006. Correcting sample selection bias by unlabeled data. In NIPS.

Digital Library

[24]

Ioffe, S., and Szegedy, C. 2015. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In ICML.

[25]

Jiang, J., and Zhai, C. 2007. Instance Weighting for Domain Adaptation in NLP. In ACL.

[26]

Krizhevsky, A.; Sutskever, I.; and Hinton, G. E. 2012. Imagenet classification with deep convolutional neural networks. In NIPS.

[27]

Kulis, B.; Saenko, K.; and Darrell, T. 2011. What you saw is not what you get: Domain adaptation using asymmetric kernel transforms. In CVPR.

Digital Library

[28]

Long, M.; Wang, J.; Ding, G.; Sun, J.; and Yu, P. 2014. Transfer joint matching for unsupervised domain adaptation. In CVPR.

Digital Library

[29]

Long, M.; Cao, Y.; Wang, J.; and Jordan, M. I. 2015. Learning transferable features with deep adaptation networks. In ICML.

[30]

Mahendran, A., and Vedaldi, A. 2015. Understanding deep image representations by inverting them. In CVPR.

[31]

Pan, S. J.; Tsang, I. W.; Kwok, J. T.; and Yang, Q. 2009. Domain adaptation via transfer component analysis. In IJCAI.

Digital Library

[32]

Saenko, K.; Kulis, B.; Fritz, M.; and Darrell, T. 2010. Adapting visual category models to new domains. In ECCV.

Digital Library

[33]

Shekhar, S.; Patel, V. M.; Nguyen, H. V.; and Chellappa, R. 2013. Generalized domain-adaptive dictionaries. In CVPR.

Digital Library

[34]

Sun, B., and Saenko, K. 2014. From virtual to reality: Fast adaptation of virtual object detectors to real domains. In BMVC.

[35]

Tommasi, T., and Caputo, B. 2013. Frustratingly easy NBNN domain adaptation. In ICCV.

Digital Library

[36]

Tommasi, T., and Tuytelaars, T. 2014. A testbed for crossdataset analysis. In ECCV TASK-CV Workshop.

[37]

Torralba, A., and Efros, A. A. 2011. Unbiased look at dataset bias. In CVPR.

Digital Library

[38]

Tzeng, E.; Hoffman, J.; Zhang, N.; Saenko, K.; and Darrell, T. 2014. Deep domain confusion: Maximizing for domain invariance. CoRR abs/1412.3474.

[39]

Xiao, J.; Hays, J.; Ehinger, K. A.; Oliva, A.; and Torralba, A. 2010. Sun database: Large-scale scene recognition from abbey to zoo. In CVPR.

[40]

Yang, J.; Yan, R.; and Hauptmann, A. 2007. Adapting SVM classifiers to data with shifted distributions. In ICDM Workshop.

Digital Library

Cited By

Allard TAsghar HAvoine GBobineau CCauchois PFromont EMonreale ANaretto FPellungrini RPratesi FRousset MVoyez A(2024)Analyzing and explaining privacy risks on time series data: ongoing work and challengesACM SIGKDD Explorations Newsletter10.1145/3682112.368211826:1(49-58)Online publication date: 25-Jul-2024
https://dl.acm.org/doi/10.1145/3682112.3682118
Zheng YCai YCai ZFan CWang SWang JSerra ESpezzano F(2024)FGITrans: Cross-City Transformer for Fine-grained Urban Flow InferenceProceedings of the 33rd ACM International Conference on Information and Knowledge Management10.1145/3627673.3679855(3415-3423)Online publication date: 21-Oct-2024
https://dl.acm.org/doi/10.1145/3627673.3679855
Gentner NSusto G(2024)Heterogeneous domain adaptation and equipment matchingComputers and Industrial Engineering10.1016/j.cie.2023.109821187:COnline publication date: 12-Apr-2024
https://dl.acm.org/doi/10.1016/j.cie.2023.109821
Show More Cited By

Return of frustratingly easy domain adaptation
1. Computing methodologies

Recommendations

Frustratingly easy semi-supervised domain adaptation
DANLP 2010: Proceedings of the 2010 Workshop on Domain Adaptation for Natural Language Processing

In this work, we propose a semisupervised extension to a well-known supervised domain adaptation approach (EA) (Daumé III, 2007). Our proposed approach (EA++) builds on the notion of augmented space (introduced in EA) and harnesses unlabeled data in ...
Frustratingly Easy NBNN Domain Adaptation
ICCV '13: Proceedings of the 2013 IEEE International Conference on Computer Vision

Over the last years, several authors have signaled that state of the art categorization methods fail to perform well when trained and tested on data from different databases. The general consensus in the literature is that this issue, known as domain ...
Asymmetric tri-training for unsupervised domain adaptation
ICML'17: Proceedings of the 34th International Conference on Machine Learning - Volume 70

It is important to apply models trained on a large number of labeled samples to different domains because collecting many labeled samples in various domains is expensive. To learn discriminative representations for the target domain, we assume that ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Guide Proceedings

AAAI'16: Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence

February 2016

4406 pages

Sponsors

Association for the Advancement of Artificial Intelligence

Publisher

AAAI Press

Publication History

Published: 12 February 2016

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

58
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Allard TAsghar HAvoine GBobineau CCauchois PFromont EMonreale ANaretto FPellungrini RPratesi FRousset MVoyez A(2024)Analyzing and explaining privacy risks on time series data: ongoing work and challengesACM SIGKDD Explorations Newsletter10.1145/3682112.368211826:1(49-58)Online publication date: 25-Jul-2024
https://dl.acm.org/doi/10.1145/3682112.3682118
Zheng YCai YCai ZFan CWang SWang JSerra ESpezzano F(2024)FGITrans: Cross-City Transformer for Fine-grained Urban Flow InferenceProceedings of the 33rd ACM International Conference on Information and Knowledge Management10.1145/3627673.3679855(3415-3423)Online publication date: 21-Oct-2024
https://dl.acm.org/doi/10.1145/3627673.3679855
Gentner NSusto G(2024)Heterogeneous domain adaptation and equipment matchingComputers and Industrial Engineering10.1016/j.cie.2023.109821187:COnline publication date: 12-Apr-2024
https://dl.acm.org/doi/10.1016/j.cie.2023.109821
Wang ZWang HOh ANaumann TGloberson ASaenko KHardt MLevine S(2023)Variational imbalanced regressionProceedings of the 37th International Conference on Neural Information Processing Systems10.5555/3666122.3667448(30429-30452)Online publication date: 10-Dec-2023
https://dl.acm.org/doi/10.5555/3666122.3667448
Ramé AAhuja KZhang JCord MBottou LLopez-Paz DKrause ABrunskill ECho KEngelhardt BSabato SScarlett J(2023)Model ratatouilleProceedings of the 40th International Conference on Machine Learning10.5555/3618408.3619598(28656-28679)Online publication date: 23-Jul-2023
https://dl.acm.org/doi/10.5555/3618408.3619598
Lv YMa JZhang YXu GFrommholz IHopfgartner FLee MOakes MLalmas MZhang MSantos R(2023)Integrating Priors into Domain Adaptation Based on Evidence TheoryProceedings of the 32nd ACM International Conference on Information and Knowledge Management10.1145/3583780.3614935(1726-1735)Online publication date: 21-Oct-2023
https://dl.acm.org/doi/10.1145/3583780.3614935
Yang SWang YWang KJui Svan de Weijer JKoyejo SMohamed SAgarwal ABelgrave DCho KOh A(2022)Attracting and dispersingProceedings of the 36th International Conference on Neural Information Processing Systems10.5555/3600270.3600690(5802-5815)Online publication date: 28-Nov-2022
https://dl.acm.org/doi/10.5555/3600270.3600690
Chen YRosenfeld ESellke MMa TRisteski AKoyejo SMohamed SAgarwal ABelgrave DCho KOh A(2022)Iterative feature matchingProceedings of the 36th International Conference on Neural Information Processing Systems10.5555/3600270.3600396(1725-1736)Online publication date: 28-Nov-2022
https://dl.acm.org/doi/10.5555/3600270.3600396
Gao PLi JZhao GDing C(2022)Multisource Deep Transfer Learning Based on Balanced Distribution AdaptationComputational Intelligence and Neuroscience10.1155/2022/69152162022Online publication date: 1-Jan-2022
https://dl.acm.org/doi/10.1155/2022/6915216
Gao PLi JDing C(2022)Multisource Mobile Transfer Learning Algorithm Based on Dynamic Model CompressionSecurity and Communication Networks10.1155/2022/32340782022Online publication date: 1-Jan-2022
https://dl.acm.org/doi/10.1155/2022/3234078
Show More Cited By

View Options

View options

Figures

Tables

Media

View Table of Conten