research-article

Domain adaptation from multiple sources via auxiliary classifiers

Authors:

Tat-Seng ChuaAuthors Info & Claims

ICML '09: Proceedings of the 26th Annual International Conference on Machine Learning

Pages 289 - 296

https://doi.org/10.1145/1553374.1553411

Published: 14 June 2009 Publication History

Abstract

We propose a multiple source domain adaptation method, referred to as Domain Adaptation Machine (DAM), to learn a robust decision function (referred to as target classifier) for label prediction of patterns from the target domain by leveraging a set of pre-computed classifiers (referred to as auxiliary/source classifiers) independently learned with the labeled patterns from multiple source domains. We introduce a new data-dependent regularizer based on smoothness assumption into Least-Squares SVM (LS-SVM), which enforces that the target classifier shares similar decision values with the auxiliary classifiers from relevant source domains on the unlabeled patterns of the target domain. In addition, we employ a sparsity regularizer to learn a sparse target classifier. Comprehensive experiments on the challenging TRECVID 2005 corpus demonstrate that DAM outperforms the existing multiple source domain adaptation methods for video concept detection in terms of effectiveness and efficiency.

References

[1]

Belkin, M., Niyogi, P., & Sindhwani, V. (2006). Manifold regularization: A geometric framework for learning from labeled and unlabeled examples. Journal of Machine Learning Research, 12, 2399--2434.

Digital Library

[2]

Blitzer, J., McDonald, R., & Pereira, F. (2006). Domain adaptation with structural correspondence learning. Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing, 120--128.

Digital Library

[3]

Borgwardt, K. M., Gretton, A., Rasch, M. J., Kriegel, H.-P., Schölkopf, B., & Smola, A. J. (2006). Integrating structured biological data by kernel maximum mean discrepancy. Proceedings of the 14th International Conference on Intelligent Systems for Molecular Biology, 49--57.

Digital Library

[4]

Crammer, K., Kearns, M., & Wortman, J. (2008). Learning from multiple sources. Journal of Machine Learning Research, 9, 1757--1774.

Digital Library

[5]

Daumé III, H. (2007). Frustratingly easy domain adaptation. Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics, 256--263.

[6]

Evgeniou, T., Micchelli, C., & Pontil, M. (2005). Learnng multiple tasks with kernel methods. Journal of Machine Learning Research, 6, 615--637.

Digital Library

[7]

Huang, J., Smola, A., Gretton, A., Borgwardt, K. M., & Schöölkopf, B. (2007). Correcting sample selection bias by unlabeled data. Advances in Neural Information Processing Systems 19, 601--608.

[8]

Joachims, T. (1999). Transductive inference for text classification using support vector machines. Proceedings of the Sixteenth International Conference on Machine Learning, 200--209.

Digital Library

[9]

Kato, T., Kashima, H., Sugiyama, M., & Asai, K. (2008). Multi-task learning via conic programming. Advances in Neural Information Processing Systems 20, 737--744.

[10]

Luo, P., Zhuang, F., Xiong, H., Xiong, Y., & He, Q. (2008). Transfer learning from multiple source domains via consensus regularization. Proceeding of the ACM 17th Conference on Information and Knowledge Management, 103--112.

Digital Library

[11]

Mansour, Y., Mohri, M., & Rostamizadeh, A. (2009). Domain adaptation with multiple sources. Advances in Neural Information Processing Systems 21, 1041--1048.

[12]

Naphade, M. R., Kennedy, L., Kender, J. R., Chang, S.-F., Smith, J. R., Over, P., & Hauptmann, A. (2005). A light scale concept ontology for multimedia understanding for TRECVID 2005 (Technical Report). IBM Research Technical Report.

[13]

Rosenstein, M. T., Marx, Z., & Kaelbling, L. P. (2005). To transfer or not to transfer. NIPS 2005 Workshop on Inductive Transfer: 10 Years Later.

[14]

Schweikert, G., Widmer, C., Schölkopf, B., & Räätsch, G. (2009). An empirical analysis of domain adaptation algorithm for genomic sequence analysis. Advances in Neural Information Processing Systems 21, 1433--1440.

[15]

Sindhwani, V., Niyogi, P., & Belkin, M. (2005). Beyond the point cloud: from transductive to semi-supervised learning. Proceedings of the Twenty-second International Conference on Machine Learning, 824--831.

Digital Library

[16]

Smeaton, A. F., Over, P., & Kraaij, W. (2006). Evaluation campaigns and trecvid. ACM International Workshop on Multimedia Information Retrieval, 321--330.

Digital Library

[17]

Storkey, A., & Sugiyama, M. (2007). Mixture regression for covariate shift. Advances in Neural Information Processing Systems 19, 1337--1344.

[18]

Van Gestel, T., Suykens, J., Baesens, B., Viaene, S., Vanthienen, J., Dedene, G., de Moor, B., & Vandewalle, J. (2004). Benchmarking least squares support vector machine classifiers. Machine Learning, 54, 5--32.

Digital Library

[19]

Wu, P., & Dietterich, T. G. (2004). Improving SVM accuracy by training on auxiliary data sources. Proceedings of the Twenty-first International Conference on Machine Learning, 871--878.

Digital Library

[20]

Yang, J., Yan, R., & Hauptmann, A. G. (2007). Cross-domain video concept detection using adaptive SVMs. Proceedings of the 15th International Conference on Multimedia, 188--197.

Digital Library

Cited By

Székely ATörök BKiss MJanacsek KNémeth DOrbán G(2024)Identifying Transfer Learning in the Reshaping of Inductive BiasesOpen Mind10.1162/opmi_a_001588(1107-1128)Online publication date: 15-Sep-2024
https://doi.org/10.1162/opmi_a_00158
Tan ZLiu JBi XTan PZheng QLiu HXie YZou XYu YZhou ZBaeza-Yates RBonchi F(2024)Beimingwu: A Learnware Dock SystemProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3671617(5773-5782)Online publication date: 25-Aug-2024
https://dl.acm.org/doi/10.1145/3637528.3671617
Pei CWu FYang MPan LDing WDong JHuang LZhuang X(2024)Multi-Source Domain Adaptation for Medical Image SegmentationIEEE Transactions on Medical Imaging10.1109/TMI.2023.334628543:4(1640-1651)Online publication date: Apr-2024
https://doi.org/10.1109/TMI.2023.3346285
Show More Cited By

Index Terms

Domain adaptation from multiple sources via auxiliary classifiers

Recommendations

Cross-Domain Learning from Multiple Sources: A Consensus Regularization Perspective

Classification across different domains studies how to adapt a learning model from one domain to another domain which shares similar data characteristics. While there are a number of existing works along this line, many of them are only focused on ...
Universal domain adaptation from multiple black-box sources
Abstract
Black-box domain adaptation (B2DA) is a practical unsupervised domain adaptation (UDA) setting, in which only an interface to source model is available due to privacy or security concerns. It aims to transfer source knowledge with no source data ...
Highlights
- Uni-MBDA is proposed for multiple black-box sources with label shift from target.
- Um2B is proposed for Uni-MBDA with dual-attention on source domains and classes.
- Empirical results show the effectiveness of Um2B for Uni-MBDA.
A two-stage approach to domain adaptation for statistical classifiers
CIKM '07: Proceedings of the sixteenth ACM conference on Conference on information and knowledge management

In this paper, we consider the problem of adapting statistical classifiers trained from some source domains where labeled examples are available to a target domain where no labeled example is available. One characteristic of such a domain adaptation ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

ICML '09: Proceedings of the 26th Annual International Conference on Machine Learning

June 2009

1331 pages

ISBN:9781605585161

DOI:10.1145/1553374

General Chair:
Andrea Danyluk
Williams College
,
Program Chairs:
Léon Bottou
NEC Laboratories America
,
Michael Littman
Rutgers University

Copyright © 2009 Copyright 2009 by the author(s)/owner(s).

Sponsors

NSF
Microsoft Research: Microsoft Research
MITACS

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 14 June 2009

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article

Funding Sources

Conference

ICML '09

Sponsor:

Microsoft Research

ICML '09: The 26th Annual International Conference on Machine Learning held in conjunction with the 2007 International Conference on Inductive Logic Programming

June 14 - 18, 2009

Quebec, Montreal, Canada

Acceptance Rates

Overall Acceptance Rate 140 of 548 submissions, 26%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

202
Total Citations
View Citations
1,374
Total Downloads

Downloads (Last 12 months)84
Downloads (Last 6 weeks)7

Reflects downloads up to 14 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Székely ATörök BKiss MJanacsek KNémeth DOrbán G(2024)Identifying Transfer Learning in the Reshaping of Inductive BiasesOpen Mind10.1162/opmi_a_001588(1107-1128)Online publication date: 15-Sep-2024
https://doi.org/10.1162/opmi_a_00158
Tan ZLiu JBi XTan PZheng QLiu HXie YZou XYu YZhou ZBaeza-Yates RBonchi F(2024)Beimingwu: A Learnware Dock SystemProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3671617(5773-5782)Online publication date: 25-Aug-2024
https://dl.acm.org/doi/10.1145/3637528.3671617
Pei CWu FYang MPan LDing WDong JHuang LZhuang X(2024)Multi-Source Domain Adaptation for Medical Image SegmentationIEEE Transactions on Medical Imaging10.1109/TMI.2023.334628543:4(1640-1651)Online publication date: Apr-2024
https://doi.org/10.1109/TMI.2023.3346285
Gaw NLi JYoon H(2024)A Novel Semi-Supervised Learning Model for Smartphone-Based Health TelemonitoringIEEE Transactions on Automation Science and Engineering10.1109/TASE.2022.321813221:1(428-441)Online publication date: Jan-2024
https://doi.org/10.1109/TASE.2022.3218132
Jin CZheng HYu H(2024)How to Design or Learn Prompt for Domain Adaptation?2024 International Joint Conference on Neural Networks (IJCNN)10.1109/IJCNN60899.2024.10651258(1-8)Online publication date: 30-Jun-2024
https://doi.org/10.1109/IJCNN60899.2024.10651258
Alevizaki APham NTrigoni N(2024)Revealing semantic mappings across HAR datasets2024 20th International Conference on Distributed Computing in Smart Systems and the Internet of Things (DCOSS-IoT)10.1109/DCOSS-IoT61029.2024.00023(93-100)Online publication date: 29-Apr-2024
https://doi.org/10.1109/DCOSS-IoT61029.2024.00023
Hinduja SNourivandi TCohn JCanavan S(2024)Time to retire F1-binary score for action unit detectionPattern Recognition Letters10.1016/j.patrec.2024.04.016Online publication date: Apr-2024
https://doi.org/10.1016/j.patrec.2024.04.016
Liu JZhang H(2024)Fast Implementation of Object Detection Algorithm Based on Homomorphic Model TransformationNeurocomputing10.1016/j.neucom.2024.127313(127313)Online publication date: Feb-2024
https://doi.org/10.1016/j.neucom.2024.127313
Ma ZLi HWeng YFarantatos E(2024)Power system transfer learning to identify event types with or without event labelsInternational Journal of Electrical Power & Energy Systems10.1016/j.ijepes.2023.109642155(109642)Online publication date: Jan-2024
https://doi.org/10.1016/j.ijepes.2023.109642
Amrish Shwetank (2024)Comparative analysis of manual and annotations for crowd assessment and classification using artificial intelligenceData Science and Management10.1016/j.dsm.2024.04.001Online publication date: Apr-2024
https://doi.org/10.1016/j.dsm.2024.04.001
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents