Abstract
Traditional machine learning requires good tags to obtain excellent performance, while manual tagging usually consumes a lot of time and money. Due to the influence of domain shift, using the trained model on the source domain directly on the target domain is not good. Domain adaptation is used to solve the above problems. The deep domain adaptation method uses deep neural networks to complete domain adaptation. This article has carried out a comprehensive review of the deep domain adaptation method of image classification. The main contributions are the following four aspects. Firstly, we divided the deep domain adaptation into several categories based on the label set of the source domain and the target domain. Secondly, we summarized various methods of Closed-set domain adaptation. Thirdly, we discussed current methods of multi-source domain adaptation. Finally, we discussed future research directions, challenges, and possible solutions.
Similar content being viewed by others
References
Ahmed A, Yousif H, He Z (2021) Ensemble diversified learning for image classification with noisy labels. Multimed Tools and Appl. https://doi.org/10.1007/s11042-021-10760-z
Alyafeai Z, Ghouti L (2020) A fully-automated deep learning pipeline for cervical cancer classification. Expert Syst Appl 141:112951. https://doi.org/10.1016/j.eswa.2019.112951
Aquino G, Rubio JDJ, Pacheco J, Gutierrez GJ, Ochoa G, Balcazar R, Cruz DR, Garcia E, Novoa JF, Zacarias A (2020) Novel nonlinear hypothesis for the delta parallel robot modeling. IEEE Access 8:46324–46334. https://doi.org/10.1109/ACCESS.2020.2979141
Ben-David S, Blitzer J, Crammer K, Kulesza A, Pereira F, Vaughan JW (2010) A theory of learning from different domains. Mach Learn 79(1):151–175
Bousmalis K, Silberman N, Dohan D, Erhan D, Krishnan D (2017) Unsupervised pixel-level domain adaptation with generative adversarial networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3722–3731
Bousmalis K, Trigeorgis G, Silberman N, Krishnan D, Erhan D (2016) Domain separation networks. In: advances in neural information processing systems, pp 343–351
Bruzzone L, Marconcini M (2009) Domain adaptation problems: A DASVM classification technique and a circular validation strategy. IEEE Trans Pattern Anal Mach Intell 32(5):770–787
Cai Z, Han J, Liu L, Shao L (2017) RGB-D datasets using microsoft kinect or similar sensors: A survey. Multimed Tools Appl 76(3):4313–4355. https://doi.org/10.1007/s11042-016-3374-6
Cai Z, Jing X-Y, Shao L (2020) Visual-depth matching network: deep rgb-D domain adaptation with unequal categories. IEEE Trans Cybern:1–13. https://doi.org/10.1109/TCYB.2020.3032194
Cai Z, Long Y, Shao L (2018) Adaptive RGB image recognition by visual-depth embedding. IEEE Trans Image Process 27(5):2471–2483. https://doi.org/10.1109/TIP.2018.2806839
Cao Z, Long M, Wang J, Jordan MI (2018) Partial transfer learning with selective adversarial networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2724–2732
Cao Z, Ma L, Long M, Wang J (2018) Partial adversarial domain adaptation. In: Proceedings of the European conference on computer vision, pp 135–150
Cao Z, You K, Long M, Wang J, Yang Q (2019) Learning to transfer examples for partial domain adaptation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2985–2994
Chakraborty S, Mondal R, Singh PK, Sarkar R, Bhattacharjee D (2021) Transfer learning with fine tuning for human action recognition from still images. Multimed Tools Appl. https://doi.org/10.1007/s11042-021-10753-y
Chang W-G, You T, Seo S, Kwak S, Han B (2019) Domain-specific batch normalization for unsupervised domain adaptation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7354–7362
Chen C, Fu Z, Chen Z, Jin S, Cheng Z, Jin X, Hua X-S (2020) HoMM: Higher-order moment matching for unsupervised domain adaptation. Order 1(10):20
Chen M, Zhao S, Liu H, Cai D (2020) Adversarial-learned loss for domain adaptation.. In: AAAI, pp 3521–3528
Chiang H-S, Chen M-Y, Huang Y-J (2019) Wavelet-Based EEG Processing for Epilepsy Detection Using Fuzzy Entropy and Associative Petri Net. IEEE Access 7:103255–103262. https://doi.org/10.1109/ACCESS.2019.2929266
Chu C, Wang R (2018) A survey of domain adaptation for neural machine translation. arXiv:1806.00258
Chu W-S, De la Torre F, Cohn JF (2013) Selective transfer machine for personalized facial action unit detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3515–3522
Csurka G (2017) Domain adaptation for visual applications: A comprehensive survey. arXiv:1702.05374
de Jesus Rubio J (2009) SOFMLS: online self-organizing fuzzy modified least-squares network. IEEE Trans Fuzzy Syst 17(6):1296–1309. https://doi.org/10.1109/TFUZZ.2009.2029569
de Rubio JJ (2020) Stability analysis of the modified levenberg-marquardt algorithm for the artificial neural network training. IEEE Trans Neural Netw Learn Syst:1–15. https://doi.org/10.1109/TNNLS.2020.3015200
Dou Q, Ouyang C, Chen C, Chen H, Heng P-A (2018) Unsupervised cross-modality domain adaptation of convnets for biomedical image segmentations with adversarial loss. arXiv:1804.1091
Fang X, Bai H, Guo Z, Shen B, Hoi S, Xu Z (2020) Dart: Domain-adversarial residual-transfer networks for unsupervised cross-domain image classification. Neural Netw
Frome A, Corrado GS, Shlens J, Bengio S, Dean J, Ranzato M, Mikolov T (2013) Devise: A deep visual-semantic embedding model. In: advances in neural information processing systems, pp 2121–2129
Fu B, Cao Z, Long M, Wang J (2020) Learning to detect open classes for universal domain adaptation. In: european conference on computer vision. Springer, pp 567–583
Ganin Y, Lempitsky V (2015) Unsupervised domain adaptation by backpropagation. In: international conference on machine learning. PMLR, pp 1180–1189
Gatys LA, Ecker AS, Bethge M (2016) Image style transfer using convolutional neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2414–2423
Gehring J, Miao Y, Metze F, Waibel A (2013) Extracting deep bottleneck features using stacked auto-encoders. In: 2013 IEEE International Conference on Acoustics, Speech and Signal Processing. IEEE, pp 3377–3381
Ghifary M, Kleijn WB, Zhang M, Balduzzi D, Li W (2016) Deep reconstruction-classification networks for unsupervised domain adaptation. In: european conference on computer vision. Springer, pp 597–613
Gong B, Grauman K, Sha F (2013) Connecting the dots with landmarks: Discriminatively learning domain-invariant features for unsupervised domain adaptation. In: international conference on machine learning, pp 222–230
Gong B, Shi Y, Sha F, Grauman K (2012) Geodesic flow kernel for unsupervised domain adaptation. In: 2012 IEEE conference on computer vision and pattern recognition. IEEE, pp 2066–2073
Gretton A, Borgwardt KM, Rasch MJ, Schölkopf B, Smola A (2012) A kernel two-sample test. J Mach Learn Res 13(1):723–773
Gulrajani I, Ahmed F, Arjovsky M, Dumoulin V, Courville AC (2017) Improved training of wasserstein gans. In: advances in neural information processing systems, pp 5767–5777
Hernández G, Zamora E, Sossa H, Téllez G, Furlán F (2020) Hybrid neural networks for big data classification. Neurocomputing 390:327–340. https://doi.org/10.1016/j.neucom.2019.08.095
Huang Z, Wang X, Huang L, Huang C, Wei Y, Liu W (2019) Ccnet: Criss-cross attention for semantic segmentation. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 603–612
Huh J-H, Seo Y-S (2019) Understanding edge computing: engineering evolution with artificial intelligence. IEEE Access 7:164229–164245. https://doi.org/10.1109/ACCESS.2019.2945338
Iwendi C, Srivastava G, Khan S, Maddikunta PKR (2020) Cyberbullying detection solutions based on deep learning architectures. Multimed Syst. https://doi.org/10.1007/s00530-020-00701-5
Kim Y (2014) Convolutional neural networks for sentence classification. arXiv:1408.5882
Kong T, Sun F, Liu H, Jiang Y, Li L, Shi J (2020) Foveabox: Beyound anchor-based object detection. IEEE Trans Image Process 29:7389–7398
Kouw WM, Loog M (2018) An introduction to domain adaptation and transfer learning. arXiv:1812.11806
Krizhevsky A, Sutskever I, Hinton GE (2017) Imagenet classification with deep convolutional neural networks. Commun ACM 60(6):84–90
Kutia S, Chauhdary SH, Iwendi C, Liu L, Yong W, Bashir AK (2019) Socio-technological factors affecting user’s adoption of eHealth functionalities: a case study of China and Ukraine eHealth systems. IEEE Access 7:90777–90788. https://doi.org/10.1109/ACCESS.2019.2924584
Larsen ABL, Sønderby SK, Larochelle H, Winther O (2016) Autoencoding beyond pixels using a learned similarity metric. In: international conference on machine learning. PMLR, pp 1558–1566
LeCun Y et al (2015) LeNet-5, convolutional neural networks. http://yann.lecun.com/exdb/lenet 20(5):14
Lee H, Park S-H, Yoo J-H, Jung S-H, Huh J-H (2020) Face recognition at a distance for a stand-alone access control system. Sensors 20(3):785. https://doi.org/10.3390/s20030785
Li S, Liu CH, Lin Q, Xie B, Ding Z, Huang G, Tang J (2020) Domain conditioned adaptation network.. In: AAAI, pp 11386–11393
Li W, Zhu X, Gong S (2018) Harmonious attention network for person re-identification. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2285–2294
Liu H, Cao Z, Long M, Wang J, Yang Q (2019) Separate to adapt: Open set domain adaptation via progressive separation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2927–2936
Liu S, Long M, Wang J, Jordan MI (2018) Generalized zero-shot learning with deep calibration network. In: advances in neural information processing systems, pp 2005–2015
Long M, Cao Y, Wang J, Jordan M (2015) Learning transferable features with deep adaptation networks. In: international conference on machine learning. PMLR, pp 97–105
Long M, Zhu H, Wang J, Jordan MI (2017) Deep transfer learning with joint adaptation networks. In: international conference on machine learning. PMLR, pp 2208–2217
Ma X, Zhang T, Xu C (2019) Deep multi-modality adversarial networks for unsupervised domain adaptation. IEEE Trans Multimed 21(9):2419–2431
Mansour Y, Mohri M, Rostamizadeh A (2009) Domain adaptation with multiple sources. In: advances in neural information processing systems, pp 1041–1048
Meda-Campana JA (2018) On the estimation and control of nonlinear systems with parametric uncertainties and noisy outputs. IEEE Access 6:31968–31973. https://doi.org/10.1109/ACCESS.2018.2846483
Mohamed A-r, Dahl GE, Hinton G (2011) Acoustic modeling using deep belief networks. IEEE Trans Audio Speech Lang Process 20(1):14–22
Mohamed A-, Hinton G, Penn G (2012) Understanding how deep belief networks perform acoustic modelling. In: 2012 IEEE international conference on acoustics, speech and signal processing. IEEE, pp 4273–4276
Narang SR, Kumar M, Jindal MK (2021) DeepNetDevanagari: A deep learning model for Devanagari ancient character recognition. Multimed Tools Appl. https://doi.org/10.1007/s11042-021-10775-6
Norouzi M, Mikolov T, Bengio S, Singer Y, Shlens J, Frome A, Corrado GS, Dean J (2013) Zero-shot learning by convex combination of semantic embeddings. arXiv:1312.5650
Pan SJ, Tsang IW, Kwok JT, Yang Q (2010) Domain adaptation via transfer component analysis. IEEE Trans Neural Netw 22(2):199–210
Pan SJ, Yang Q (2009) A survey on transfer learning. IEEE Trans Knowl Data Eng 22(10):1345–1359
Panareda Busto P, Gall J (2017) Open set domain adaptation. In: Proceedings of the IEEE international conference on computer vision, pp 754–763
Patel VM, Gopalan R, Li R, Chellappa R (2015) Visual domain adaptation: A survey of recent advances. IEEE Signal Process Mag 32(3):53–69
Pei Z, Cao Z, Long M, Wang J (2018) Multi-adversarial domain adaptation. arXiv:1809.02176
Peng K-C, Wu Z, Ernst J (2018) Zero-shot deep domain adaptation. In: Proceedings of the European conference on computer vision, pp 764–781
Peng X, Bai Q, Xia X, Huang Z, Saenko K, Wang B (2019) Moment matching for multi-source domain adaptation. In: Proceedings of the IEEE international conference on computer vision, pp 1406–1415
Peng X, Usman B, Kaushik N, Hoffman J, Wang D, Saenko K (2017) Visda: The visual domain adaptation challenge. arXiv:1710.06924
Rakshit RD, Kisku DR, Gupta P, Sing JK (2021) Cross-resolution face identification using deep-convolutional neural network. Multimed Tools Appl. https://doi.org/10.1007/s11042-021-10745-y
Saenko K, Kulis B, Fritz M, Darrell T (2010) Adapting visual category models to new domains. In: European conference on computer vision. Springer, pp 213–226
Saito K, Kim D, Sclaroff S, Darrell T, Saenko K (2019) Semi-supervised domain adaptation via minimax entropy. In: Proceedings of the IEEE international conference on computer vision, pp 8050–8058
Saito K, Kim D, Sclaroff S, Saenko K (2020) Universal domain adaptation through self supervision. arXiv:2002.07953
Saito K, Ushiku Y, Harada T, Saenko K (2017) Adversarial dropout regularization. arXiv:1711.01575
Saito K, Yamamoto S, Ushiku Y, Harada T (2018) Open set domain adaptation by backpropagation. In: Proceedings of the European conference on computer vision, pp 153–168
Sankaranarayanan S, Balaji Y, Castillo CD, Chellappa R (2018) Generate to adapt: Aligning domains using generative adversarial networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 8503–8512
Seo Y-S, Huh J-H (2019) Automatic emotion-based music classification for supporting intelligent IoT applications. Electronics 8(2):164. https://doi.org/10.3390/electronics8020164
Shen J, Qu Y, Zhang W, Yu Y (2017) Wasserstein distance guided representation learning for domain adaptation. arXiv:1707.01217
Shi H, Lin G, Wang H, Hung T-Y, Wang Z (2020) Spsequencenet: Semantic segmentation network on 4d point clouds. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 4574–4583
Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556
Sothmann T, Gauer T, Werner R (2018) Influence of 4d ct motion artifacts on correspondence model-based 4d dose accumulation. In: medical imaging 2018: image-guided procedures, robotic interventions, and modeling, vol 10576. International Society for Optics and Photonics, p 105760F
Sun B, Saenko K (2016) Deep coral: Correlation alignment for deep domain adaptation. In: European conference on computer vision. Springer, pp 443–450
Sun S, Shi H, Wu Y (2015) A survey of multi-source domain adaptation. Inf Fusion 24:84–92
Syed AM, Anjum A, Khan S, Mohan S, Srivastava G (2020) N-Sanitization: A semantic privacy-preserving framework for unstructured medical datasets. Comput Commun 161:160–171. https://doi.org/10.1016/j.comcom.2020.07.032
Tan M, Pang R, Le QV (2020) Efficientdet: Scalable and efficient object detection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 10781–10790
Tang H, Jia K (April 2020) Discriminative adversarial domain adaptation. Proc AAAI Conf Artif Intell 34(04):5940–5947. https://doi.org/10.1609/aaai.v34i04.6054
Tzeng E, Hoffman J, Saenko K, Darrell T (2017) Adversarial discriminative domain adaptation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7167–7176
Tzeng E, Hoffman J, Zhang N, Saenko K, Darrell T (2014) Deep domain confusion: Maximizing for domain invariance. arXiv:1412.3474
Van Engelen JE, Hoos HH (2020) A survey on semi-supervised learning. Mach Learn 109(2):373–440
Venkateswara H, Eusebio J, Chakraborty S, Panchanathan S (2017) Deep hashing network for unsupervised domain adaptation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5018–5027
Vincent P, Larochelle H, Lajoie I, Bengio Y, Manzagol P-A, Bottou L (2010) Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion. J Mach Learn Res 11(12)
Wang J, Jiang J (2019) Conditional coupled generative adversarial networks for zero-shot domain adaptation. In: Proceedings of the IEEE international conference on computer vision, pp 3375–3384
Wang J, Jiang J (2020) Adversarial learning for zero-shot domain adaptation. In: European conference on computer vision. Springer, pp 329–344
Wang M, Deng W (2018) Deep visual domain adaptation: A survey. Neurocomputing 312:135–153
Wang W, Zheng VW, Yu H, Miao C (2019) A survey of zero-shot learning: Settings, methods, and applications. ACM Trans Intell Syst Technol 10(2):1–37
Wen J, Greiner R, Schuurmans D (2020) Domain aggregation networks for multi-source domain adaptation. In: international conference on machine learning. PMLR, pp 10214–10224
Wilson G, Cook DJ (2020) A survey of unsupervised deep domain adaptation. ACM Trans Intell Syst Technol 11(5):1–46. https://doi.org/10.1145/3400066
Xu M, Zhang J, Ni B, Li T, Wang C, Tian Q, Zhang W (2020) Adversarial domain adaptation with domain mixup. In: Proceedings of the AAAI conference on artificial intelligence, vol 34, pp 6502–6509
Xu R, Chen Z, Zuo W, Yan J, Lin L (2018) Deep cocktail network: Multi-source unsupervised domain adaptation with category shift. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3964–3973
Yang J, Yan R, Hauptmann AG (2007) Cross-domain video concept detection using adaptive svms. In: Proceedings of the 15th ACM international conference on multimedia, pp 188–197
Yosinski J, Clune J, Bengio Y, Lipson H (2014) How transferable are features in deep neural networks?. In: advances in neural information processing systems, pp 3320–3328
You K, Long M, Cao Z, Wang J, Jordan MI (2019) Universal domain adaptation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2720–2729
Zellinger W, Grubinger T, Lughofer E, Natschläger T, Saminger-Platz S (2017) Central moment discrepancy (cmd) for domain-invariant representation learning. arXiv:1702.08811
Zhai X, Oliver A, Kolesnikov A, Beyer L (2019) S4l: Self-supervised semi-supervised learning. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 1476–1485
Zhang G, Jiang T, Yang J, Xu J, Zheng Y (2021) Cross-view kernel collaborative representation classification for person re-identification. Multimed Tools Appl. https://doi.org/10.1007/s11042-021-10671-z
Zhang J, Ding Z, Li W, Ogunbona P (2018) Importance weighted adversarial nets for partial domain adaptation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 8156–8164
Zhang L, Xiang T, Gong S (2017) Learning a deep embedding model for zero-shot learning. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2021–2030
Zhang M, Hu H, Li Z, Chen J (2021) Attention-based encoder-decoder networks for workflow recognition. Multimed Tools Appl. https://doi.org/10.1007/s11042-021-10633-5
Zhang W, Xu D, Zhang J, Ouyang W (2021) Progressive modality cooperation for multi-modality domain adaptation. IEEE Trans Image Process 30:3293–3306
Zhuang F, Qi Z, Duan K, Xi D, Zhu Y, Zhu H, Xiong H, He Q (2020) A comprehensive survey on transfer learning. Proc IEEE 109(1):43–76
Acknowledgments
This work is partly supported by the Natural Science Foundation of China (Grant No. 62006127, 61833011 and 62073173), partly supported by NUPTSF under Grant NY218120 and Grant NY220021, and partly supported by Jiangsu Shuang-Chuang Project under Grant CZ005SC19019 and Nanjing Overseas Innovation Project Grant RK005NLX20001. It is also supported by National Science Foundation of Jiangsu Province, China (Grant No. BK20191376 and BK20190728).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Fan, M., Cai, Z., Zhang, T. et al. A survey of deep domain adaptation based on label set classification. Multimed Tools Appl 81, 39545–39576 (2022). https://doi.org/10.1007/s11042-022-12630-8
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-022-12630-8