Nothing Special   »   [go: up one dir, main page]

Skip to main content

Advertisement

Log in

A survey of deep domain adaptation based on label set classification

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Traditional machine learning requires good tags to obtain excellent performance, while manual tagging usually consumes a lot of time and money. Due to the influence of domain shift, using the trained model on the source domain directly on the target domain is not good. Domain adaptation is used to solve the above problems. The deep domain adaptation method uses deep neural networks to complete domain adaptation. This article has carried out a comprehensive review of the deep domain adaptation method of image classification. The main contributions are the following four aspects. Firstly, we divided the deep domain adaptation into several categories based on the label set of the source domain and the target domain. Secondly, we summarized various methods of Closed-set domain adaptation. Thirdly, we discussed current methods of multi-source domain adaptation. Finally, we discussed future research directions, challenges, and possible solutions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18

Similar content being viewed by others

References

  1. Ahmed A, Yousif H, He Z (2021) Ensemble diversified learning for image classification with noisy labels. Multimed Tools and Appl. https://doi.org/10.1007/s11042-021-10760-z

  2. Alyafeai Z, Ghouti L (2020) A fully-automated deep learning pipeline for cervical cancer classification. Expert Syst Appl 141:112951. https://doi.org/10.1016/j.eswa.2019.112951

    Article  Google Scholar 

  3. Aquino G, Rubio JDJ, Pacheco J, Gutierrez GJ, Ochoa G, Balcazar R, Cruz DR, Garcia E, Novoa JF, Zacarias A (2020) Novel nonlinear hypothesis for the delta parallel robot modeling. IEEE Access 8:46324–46334. https://doi.org/10.1109/ACCESS.2020.2979141

    Article  Google Scholar 

  4. Ben-David S, Blitzer J, Crammer K, Kulesza A, Pereira F, Vaughan JW (2010) A theory of learning from different domains. Mach Learn 79(1):151–175

    Article  MathSciNet  Google Scholar 

  5. Bousmalis K, Silberman N, Dohan D, Erhan D, Krishnan D (2017) Unsupervised pixel-level domain adaptation with generative adversarial networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3722–3731

  6. Bousmalis K, Trigeorgis G, Silberman N, Krishnan D, Erhan D (2016) Domain separation networks. In: advances in neural information processing systems, pp 343–351

  7. Bruzzone L, Marconcini M (2009) Domain adaptation problems: A DASVM classification technique and a circular validation strategy. IEEE Trans Pattern Anal Mach Intell 32(5):770–787

    Article  Google Scholar 

  8. Cai Z, Han J, Liu L, Shao L (2017) RGB-D datasets using microsoft kinect or similar sensors: A survey. Multimed Tools Appl 76(3):4313–4355. https://doi.org/10.1007/s11042-016-3374-6

    Article  Google Scholar 

  9. Cai Z, Jing X-Y, Shao L (2020) Visual-depth matching network: deep rgb-D domain adaptation with unequal categories. IEEE Trans Cybern:1–13. https://doi.org/10.1109/TCYB.2020.3032194

  10. Cai Z, Long Y, Shao L (2018) Adaptive RGB image recognition by visual-depth embedding. IEEE Trans Image Process 27(5):2471–2483. https://doi.org/10.1109/TIP.2018.2806839

    Article  MathSciNet  Google Scholar 

  11. Cao Z, Long M, Wang J, Jordan MI (2018) Partial transfer learning with selective adversarial networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2724–2732

  12. Cao Z, Ma L, Long M, Wang J (2018) Partial adversarial domain adaptation. In: Proceedings of the European conference on computer vision, pp 135–150

  13. Cao Z, You K, Long M, Wang J, Yang Q (2019) Learning to transfer examples for partial domain adaptation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2985–2994

  14. Chakraborty S, Mondal R, Singh PK, Sarkar R, Bhattacharjee D (2021) Transfer learning with fine tuning for human action recognition from still images. Multimed Tools Appl. https://doi.org/10.1007/s11042-021-10753-y

  15. Chang W-G, You T, Seo S, Kwak S, Han B (2019) Domain-specific batch normalization for unsupervised domain adaptation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7354–7362

  16. Chen C, Fu Z, Chen Z, Jin S, Cheng Z, Jin X, Hua X-S (2020) HoMM: Higher-order moment matching for unsupervised domain adaptation. Order 1(10):20

    Google Scholar 

  17. Chen M, Zhao S, Liu H, Cai D (2020) Adversarial-learned loss for domain adaptation.. In: AAAI, pp 3521–3528

  18. Chiang H-S, Chen M-Y, Huang Y-J (2019) Wavelet-Based EEG Processing for Epilepsy Detection Using Fuzzy Entropy and Associative Petri Net. IEEE Access 7:103255–103262. https://doi.org/10.1109/ACCESS.2019.2929266

    Article  Google Scholar 

  19. Chu C, Wang R (2018) A survey of domain adaptation for neural machine translation. arXiv:1806.00258

  20. Chu W-S, De la Torre F, Cohn JF (2013) Selective transfer machine for personalized facial action unit detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3515–3522

  21. Csurka G (2017) Domain adaptation for visual applications: A comprehensive survey. arXiv:1702.05374

  22. de Jesus Rubio J (2009) SOFMLS: online self-organizing fuzzy modified least-squares network. IEEE Trans Fuzzy Syst 17(6):1296–1309. https://doi.org/10.1109/TFUZZ.2009.2029569

    Article  Google Scholar 

  23. de Rubio JJ (2020) Stability analysis of the modified levenberg-marquardt algorithm for the artificial neural network training. IEEE Trans Neural Netw Learn Syst:1–15. https://doi.org/10.1109/TNNLS.2020.3015200

  24. Dou Q, Ouyang C, Chen C, Chen H, Heng P-A (2018) Unsupervised cross-modality domain adaptation of convnets for biomedical image segmentations with adversarial loss. arXiv:1804.1091

  25. Fang X, Bai H, Guo Z, Shen B, Hoi S, Xu Z (2020) Dart: Domain-adversarial residual-transfer networks for unsupervised cross-domain image classification. Neural Netw

  26. Frome A, Corrado GS, Shlens J, Bengio S, Dean J, Ranzato M, Mikolov T (2013) Devise: A deep visual-semantic embedding model. In: advances in neural information processing systems, pp 2121–2129

  27. Fu B, Cao Z, Long M, Wang J (2020) Learning to detect open classes for universal domain adaptation. In: european conference on computer vision. Springer, pp 567–583

  28. Ganin Y, Lempitsky V (2015) Unsupervised domain adaptation by backpropagation. In: international conference on machine learning. PMLR, pp 1180–1189

  29. Gatys LA, Ecker AS, Bethge M (2016) Image style transfer using convolutional neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2414–2423

  30. Gehring J, Miao Y, Metze F, Waibel A (2013) Extracting deep bottleneck features using stacked auto-encoders. In: 2013 IEEE International Conference on Acoustics, Speech and Signal Processing. IEEE, pp 3377–3381

  31. Ghifary M, Kleijn WB, Zhang M, Balduzzi D, Li W (2016) Deep reconstruction-classification networks for unsupervised domain adaptation. In: european conference on computer vision. Springer, pp 597–613

  32. Gong B, Grauman K, Sha F (2013) Connecting the dots with landmarks: Discriminatively learning domain-invariant features for unsupervised domain adaptation. In: international conference on machine learning, pp 222–230

  33. Gong B, Shi Y, Sha F, Grauman K (2012) Geodesic flow kernel for unsupervised domain adaptation. In: 2012 IEEE conference on computer vision and pattern recognition. IEEE, pp 2066–2073

  34. Gretton A, Borgwardt KM, Rasch MJ, Schölkopf B, Smola A (2012) A kernel two-sample test. J Mach Learn Res 13(1):723–773

    MathSciNet  MATH  Google Scholar 

  35. Gulrajani I, Ahmed F, Arjovsky M, Dumoulin V, Courville AC (2017) Improved training of wasserstein gans. In: advances in neural information processing systems, pp 5767–5777

  36. Hernández G, Zamora E, Sossa H, Téllez G, Furlán F (2020) Hybrid neural networks for big data classification. Neurocomputing 390:327–340. https://doi.org/10.1016/j.neucom.2019.08.095

    Article  Google Scholar 

  37. Huang Z, Wang X, Huang L, Huang C, Wei Y, Liu W (2019) Ccnet: Criss-cross attention for semantic segmentation. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 603–612

  38. Huh J-H, Seo Y-S (2019) Understanding edge computing: engineering evolution with artificial intelligence. IEEE Access 7:164229–164245. https://doi.org/10.1109/ACCESS.2019.2945338

    Article  Google Scholar 

  39. Iwendi C, Srivastava G, Khan S, Maddikunta PKR (2020) Cyberbullying detection solutions based on deep learning architectures. Multimed Syst. https://doi.org/10.1007/s00530-020-00701-5

  40. Kim Y (2014) Convolutional neural networks for sentence classification. arXiv:1408.5882

  41. Kong T, Sun F, Liu H, Jiang Y, Li L, Shi J (2020) Foveabox: Beyound anchor-based object detection. IEEE Trans Image Process 29:7389–7398

    Article  Google Scholar 

  42. Kouw WM, Loog M (2018) An introduction to domain adaptation and transfer learning. arXiv:1812.11806

  43. Krizhevsky A, Sutskever I, Hinton GE (2017) Imagenet classification with deep convolutional neural networks. Commun ACM 60(6):84–90

    Article  Google Scholar 

  44. Kutia S, Chauhdary SH, Iwendi C, Liu L, Yong W, Bashir AK (2019) Socio-technological factors affecting user’s adoption of eHealth functionalities: a case study of China and Ukraine eHealth systems. IEEE Access 7:90777–90788. https://doi.org/10.1109/ACCESS.2019.2924584

    Article  Google Scholar 

  45. Larsen ABL, Sønderby SK, Larochelle H, Winther O (2016) Autoencoding beyond pixels using a learned similarity metric. In: international conference on machine learning. PMLR, pp 1558–1566

  46. LeCun Y et al (2015) LeNet-5, convolutional neural networks. http://yann.lecun.com/exdb/lenet 20(5):14

  47. Lee H, Park S-H, Yoo J-H, Jung S-H, Huh J-H (2020) Face recognition at a distance for a stand-alone access control system. Sensors 20(3):785. https://doi.org/10.3390/s20030785

    Article  Google Scholar 

  48. Li S, Liu CH, Lin Q, Xie B, Ding Z, Huang G, Tang J (2020) Domain conditioned adaptation network.. In: AAAI, pp 11386–11393

  49. Li W, Zhu X, Gong S (2018) Harmonious attention network for person re-identification. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2285–2294

  50. Liu H, Cao Z, Long M, Wang J, Yang Q (2019) Separate to adapt: Open set domain adaptation via progressive separation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2927–2936

  51. Liu S, Long M, Wang J, Jordan MI (2018) Generalized zero-shot learning with deep calibration network. In: advances in neural information processing systems, pp 2005–2015

  52. Long M, Cao Y, Wang J, Jordan M (2015) Learning transferable features with deep adaptation networks. In: international conference on machine learning. PMLR, pp 97–105

  53. Long M, Zhu H, Wang J, Jordan MI (2017) Deep transfer learning with joint adaptation networks. In: international conference on machine learning. PMLR, pp 2208–2217

  54. Ma X, Zhang T, Xu C (2019) Deep multi-modality adversarial networks for unsupervised domain adaptation. IEEE Trans Multimed 21(9):2419–2431

    Article  Google Scholar 

  55. Mansour Y, Mohri M, Rostamizadeh A (2009) Domain adaptation with multiple sources. In: advances in neural information processing systems, pp 1041–1048

  56. Meda-Campana JA (2018) On the estimation and control of nonlinear systems with parametric uncertainties and noisy outputs. IEEE Access 6:31968–31973. https://doi.org/10.1109/ACCESS.2018.2846483

    Article  Google Scholar 

  57. Mohamed A-r, Dahl GE, Hinton G (2011) Acoustic modeling using deep belief networks. IEEE Trans Audio Speech Lang Process 20(1):14–22

    Article  Google Scholar 

  58. Mohamed A-, Hinton G, Penn G (2012) Understanding how deep belief networks perform acoustic modelling. In: 2012 IEEE international conference on acoustics, speech and signal processing. IEEE, pp 4273–4276

  59. Narang SR, Kumar M, Jindal MK (2021) DeepNetDevanagari: A deep learning model for Devanagari ancient character recognition. Multimed Tools Appl. https://doi.org/10.1007/s11042-021-10775-6

  60. Norouzi M, Mikolov T, Bengio S, Singer Y, Shlens J, Frome A, Corrado GS, Dean J (2013) Zero-shot learning by convex combination of semantic embeddings. arXiv:1312.5650

  61. Pan SJ, Tsang IW, Kwok JT, Yang Q (2010) Domain adaptation via transfer component analysis. IEEE Trans Neural Netw 22(2):199–210

    Article  Google Scholar 

  62. Pan SJ, Yang Q (2009) A survey on transfer learning. IEEE Trans Knowl Data Eng 22(10):1345–1359

    Article  Google Scholar 

  63. Panareda Busto P, Gall J (2017) Open set domain adaptation. In: Proceedings of the IEEE international conference on computer vision, pp 754–763

  64. Patel VM, Gopalan R, Li R, Chellappa R (2015) Visual domain adaptation: A survey of recent advances. IEEE Signal Process Mag 32(3):53–69

    Article  Google Scholar 

  65. Pei Z, Cao Z, Long M, Wang J (2018) Multi-adversarial domain adaptation. arXiv:1809.02176

  66. Peng K-C, Wu Z, Ernst J (2018) Zero-shot deep domain adaptation. In: Proceedings of the European conference on computer vision, pp 764–781

  67. Peng X, Bai Q, Xia X, Huang Z, Saenko K, Wang B (2019) Moment matching for multi-source domain adaptation. In: Proceedings of the IEEE international conference on computer vision, pp 1406–1415

  68. Peng X, Usman B, Kaushik N, Hoffman J, Wang D, Saenko K (2017) Visda: The visual domain adaptation challenge. arXiv:1710.06924

  69. Rakshit RD, Kisku DR, Gupta P, Sing JK (2021) Cross-resolution face identification using deep-convolutional neural network. Multimed Tools Appl. https://doi.org/10.1007/s11042-021-10745-y

  70. Saenko K, Kulis B, Fritz M, Darrell T (2010) Adapting visual category models to new domains. In: European conference on computer vision. Springer, pp 213–226

  71. Saito K, Kim D, Sclaroff S, Darrell T, Saenko K (2019) Semi-supervised domain adaptation via minimax entropy. In: Proceedings of the IEEE international conference on computer vision, pp 8050–8058

  72. Saito K, Kim D, Sclaroff S, Saenko K (2020) Universal domain adaptation through self supervision. arXiv:2002.07953

  73. Saito K, Ushiku Y, Harada T, Saenko K (2017) Adversarial dropout regularization. arXiv:1711.01575

  74. Saito K, Yamamoto S, Ushiku Y, Harada T (2018) Open set domain adaptation by backpropagation. In: Proceedings of the European conference on computer vision, pp 153–168

  75. Sankaranarayanan S, Balaji Y, Castillo CD, Chellappa R (2018) Generate to adapt: Aligning domains using generative adversarial networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 8503–8512

  76. Seo Y-S, Huh J-H (2019) Automatic emotion-based music classification for supporting intelligent IoT applications. Electronics 8(2):164. https://doi.org/10.3390/electronics8020164

    Article  Google Scholar 

  77. Shen J, Qu Y, Zhang W, Yu Y (2017) Wasserstein distance guided representation learning for domain adaptation. arXiv:1707.01217

  78. Shi H, Lin G, Wang H, Hung T-Y, Wang Z (2020) Spsequencenet: Semantic segmentation network on 4d point clouds. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 4574–4583

  79. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556

  80. Sothmann T, Gauer T, Werner R (2018) Influence of 4d ct motion artifacts on correspondence model-based 4d dose accumulation. In: medical imaging 2018: image-guided procedures, robotic interventions, and modeling, vol 10576. International Society for Optics and Photonics, p 105760F

  81. Sun B, Saenko K (2016) Deep coral: Correlation alignment for deep domain adaptation. In: European conference on computer vision. Springer, pp 443–450

  82. Sun S, Shi H, Wu Y (2015) A survey of multi-source domain adaptation. Inf Fusion 24:84–92

    Article  Google Scholar 

  83. Syed AM, Anjum A, Khan S, Mohan S, Srivastava G (2020) N-Sanitization: A semantic privacy-preserving framework for unstructured medical datasets. Comput Commun 161:160–171. https://doi.org/10.1016/j.comcom.2020.07.032

    Article  Google Scholar 

  84. Tan M, Pang R, Le QV (2020) Efficientdet: Scalable and efficient object detection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 10781–10790

  85. Tang H, Jia K (April 2020) Discriminative adversarial domain adaptation. Proc AAAI Conf Artif Intell 34(04):5940–5947. https://doi.org/10.1609/aaai.v34i04.6054

    Google Scholar 

  86. Tzeng E, Hoffman J, Saenko K, Darrell T (2017) Adversarial discriminative domain adaptation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7167–7176

  87. Tzeng E, Hoffman J, Zhang N, Saenko K, Darrell T (2014) Deep domain confusion: Maximizing for domain invariance. arXiv:1412.3474

  88. Van Engelen JE, Hoos HH (2020) A survey on semi-supervised learning. Mach Learn 109(2):373–440

    Article  MathSciNet  Google Scholar 

  89. Venkateswara H, Eusebio J, Chakraborty S, Panchanathan S (2017) Deep hashing network for unsupervised domain adaptation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5018–5027

  90. Vincent P, Larochelle H, Lajoie I, Bengio Y, Manzagol P-A, Bottou L (2010) Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion. J Mach Learn Res 11(12)

  91. Wang J, Jiang J (2019) Conditional coupled generative adversarial networks for zero-shot domain adaptation. In: Proceedings of the IEEE international conference on computer vision, pp 3375–3384

  92. Wang J, Jiang J (2020) Adversarial learning for zero-shot domain adaptation. In: European conference on computer vision. Springer, pp 329–344

  93. Wang M, Deng W (2018) Deep visual domain adaptation: A survey. Neurocomputing 312:135–153

    Article  Google Scholar 

  94. Wang W, Zheng VW, Yu H, Miao C (2019) A survey of zero-shot learning: Settings, methods, and applications. ACM Trans Intell Syst Technol 10(2):1–37

    Google Scholar 

  95. Wen J, Greiner R, Schuurmans D (2020) Domain aggregation networks for multi-source domain adaptation. In: international conference on machine learning. PMLR, pp 10214–10224

  96. Wilson G, Cook DJ (2020) A survey of unsupervised deep domain adaptation. ACM Trans Intell Syst Technol 11(5):1–46. https://doi.org/10.1145/3400066

    Article  Google Scholar 

  97. Xu M, Zhang J, Ni B, Li T, Wang C, Tian Q, Zhang W (2020) Adversarial domain adaptation with domain mixup. In: Proceedings of the AAAI conference on artificial intelligence, vol 34, pp 6502–6509

  98. Xu R, Chen Z, Zuo W, Yan J, Lin L (2018) Deep cocktail network: Multi-source unsupervised domain adaptation with category shift. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3964–3973

  99. Yang J, Yan R, Hauptmann AG (2007) Cross-domain video concept detection using adaptive svms. In: Proceedings of the 15th ACM international conference on multimedia, pp 188–197

  100. Yosinski J, Clune J, Bengio Y, Lipson H (2014) How transferable are features in deep neural networks?. In: advances in neural information processing systems, pp 3320–3328

  101. You K, Long M, Cao Z, Wang J, Jordan MI (2019) Universal domain adaptation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2720–2729

  102. Zellinger W, Grubinger T, Lughofer E, Natschläger T, Saminger-Platz S (2017) Central moment discrepancy (cmd) for domain-invariant representation learning. arXiv:1702.08811

  103. Zhai X, Oliver A, Kolesnikov A, Beyer L (2019) S4l: Self-supervised semi-supervised learning. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 1476–1485

  104. Zhang G, Jiang T, Yang J, Xu J, Zheng Y (2021) Cross-view kernel collaborative representation classification for person re-identification. Multimed Tools Appl. https://doi.org/10.1007/s11042-021-10671-z

  105. Zhang J, Ding Z, Li W, Ogunbona P (2018) Importance weighted adversarial nets for partial domain adaptation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 8156–8164

  106. Zhang L, Xiang T, Gong S (2017) Learning a deep embedding model for zero-shot learning. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2021–2030

  107. Zhang M, Hu H, Li Z, Chen J (2021) Attention-based encoder-decoder networks for workflow recognition. Multimed Tools Appl. https://doi.org/10.1007/s11042-021-10633-5

  108. Zhang W, Xu D, Zhang J, Ouyang W (2021) Progressive modality cooperation for multi-modality domain adaptation. IEEE Trans Image Process 30:3293–3306

    Article  Google Scholar 

  109. Zhuang F, Qi Z, Duan K, Xi D, Zhu Y, Zhu H, Xiong H, He Q (2020) A comprehensive survey on transfer learning. Proc IEEE 109(1):43–76

    Article  Google Scholar 

Download references

Acknowledgments

This work is partly supported by the Natural Science Foundation of China (Grant No. 62006127, 61833011 and 62073173), partly supported by NUPTSF under Grant NY218120 and Grant NY220021, and partly supported by Jiangsu Shuang-Chuang Project under Grant CZ005SC19019 and Nanjing Overseas Innovation Project Grant RK005NLX20001. It is also supported by National Science Foundation of Jiangsu Province, China (Grant No. BK20191376 and BK20190728).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Ziyun Cai.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fan, M., Cai, Z., Zhang, T. et al. A survey of deep domain adaptation based on label set classification. Multimed Tools Appl 81, 39545–39576 (2022). https://doi.org/10.1007/s11042-022-12630-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-12630-8

Keywords

Navigation