Nothing Special   »   [go: up one dir, main page]
Skip to main content
Springer Nature Link
Log in

TsCANet: Three-stream contrastive adaptive network for cross-domain few-shot learning

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

Cross-domain few-shot learning, which aims to solve the problem of domain gap in few-shot learning, has recently received more and more attention. Specifically, when there are great differences between the source domain and the target domain involved in few-shot learning, the performance will fall off a cliff and it is even difficult to train. Therefore, this paper explores a simple, effective and novel method to deal with domain gaps. Firstly, the pre-trained model is obtained by using the labeled data in the source domain. Next, the two-stage adaptive training mainly consists of unlabeled data in the target domain, pseudo-unlabeled data and labeled data in the source domain as the third-stream input, so that the network can gradually adapt to the data in target domain and mitigate the adverse effects caused by the domain gap. Finally, the proposed network can be quickly applied to the tasks to be solved. Through the observation of experimental results, the designed approach can achieve better performance than the existing comparison methods on the standard benchmark of cross-domain few-shot learning. Further analysis reveals the tradeoff between using data in source domain and target domain for cross-domain few-shot learning.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Algorithm 1
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Data availability

All the data that support the findings of this study are available from the corresponding author upon reasonable request.

References

  1. Hassantabar S, Terway P, Jha NK (2023) TUTOR: training neural networks using decision rules as model priors. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems. 42(2):483–496. https://doi.org/10.1109/TCAD.2022.3179245

    Article  Google Scholar 

  2. Guo Y, Codella N, Karlinsky L, Codella JV, Smith JR, Saenko K, Rosing T, Feris R (2020) A broader study of cross-domain few-shot learning. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J. (eds.) 16th European Conference of Computer Vision, ECCV, Glasgow, UK, August 23-28, 2020, Proceedings, Part XXVII. Lecture Notes in Computer Science, vol. 12372, pp. 124–141. https://doi.org/10.1007/978-3-030-58583-9_8

  3. Chen H, Li L, Hu F, Lyu F, Zhao L, Huang K, Feng W, Xia Z (2023) Multi-semantic hypergraph neural network for effective few-shot learning. Pattern Recognition 142:109677. https://doi.org/10.1016/j.patcog.2023.109677

    Article  Google Scholar 

  4. Shi B, Li W, Huo J, Zhu P, Wang L, Gao Y (2023) Global- and local-aware feature augmentation with semantic orthogonality for few-shot image classification. Pattern Recognition 142:109702. https://doi.org/10.1016/j.patcog.2023.109702

    Article  Google Scholar 

  5. Xie J, Long F, Lv J, Wang Q, Li P (2022) Joint distribution matters: Deep brownian distance covariance for few-shot classification. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR, New Orleans, LA, USA, June 18-24, 2022, pp. 7962–7971. https://doi.org/10.1109/CVPR52688.2022.00781

  6. Luo X, Wu H, Zhang J, Gao L, Xu J, Song J (2023) A closer look at few-shot classification again. In: International Conference on Machine Learning, ICML, 23-29 July 2023, Honolulu, Hawaii, USA. Proceedings of Machine Learning Research, vol. 202, pp. 23103–23123. https://proceedings.mlr.press/v202/luo23e.html

  7. Chen Y, Liu Z, Xu H, Darrell T, Wang X (2021) Meta-baseline: Exploring simple meta-learning for few-shot learning. In: International Conference on Computer Vision, ICCV, Montreal, QC, Canada, October 10-17, 2021, pp. 9042–9051. https://doi.org/10.1109/ICCV48922.2021.00893

  8. Chen W, Liu Y, Kira Z, Wang YF, Huang J (2019) A closer look at few-shot classification. In: 7th International Conference on Learning Representations, ICLR, New Orleans, LA, USA, May 6-9, 2019. https://openreview.net/forum?id=HkxLXnAcFQ

  9. Fu Y, Fu Y, Jiang Y (2021) Meta-fdmixup: Cross-domain few-shot learning guided by labeled target data. In: MM ’21: ACM Multimedia Conference, Virtual Event, China, October 20 - 24, 2021, pp. 5326–5334. https://doi.org/10.1145/3474085.3475655

  10. Li P, Gong S, Wang C, Fu Y (2022) Ranking distance calibration for cross-domain few-shot learning. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR, New Orleans, LA, USA, June 18-24, 2022, pp. 9089–9098. https://doi.org/10.1109/CVPR52688.2022.00889

  11. Tseng H, Lee H, Huang J, Yang M (2020) Cross-domain few-shot classification via learned feature-wise transformation. In: 8th International Conference on Learning Representations, ICLR, Addis Ababa, Ethiopia, April 26-30, 2020. https://openreview.net/forum?id=SJl5Np4tPr

  12. Wang X, Peng Y, Lu L, Lu Z, Bagheri M, Summers RM (2017) Chestx-ray8: Hospital-scale chest x-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR, Honolulu, HI, USA, July 21-26, 2017, pp. 3462–3471. https://doi.org/10.1109/CVPR.2017.369

  13. Helber P, Bischke B, Dengel A, Borth D (2019) Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification. IEEE Journal of Selection Topics in Applied Earth Observations and Remote Sensing 12(7):2217–2226. https://doi.org/10.1109/JSTARS.2019.2918242

    Article  Google Scholar 

  14. Chug A, Bhatia A, Singh AP, Singh D (2023) A novel framework for image-based plant disease detection using hybrid deep learning approach. Soft Computing 27(18):13613–13638. https://doi.org/10.1007/s00500-022-07177-7

    Article  Google Scholar 

  15. Veronica R, Halpern A, Dusza SW, Codella NCF (2019) The role of public challenges and data sets towards algorithm development, trust, and use in clinical practice. Seminars in Cutaneous Medicine and Surgery 38(1):38–42. https://doi.org/10.12788/j.sder.2019.013. PMID: 31051022

  16. Phoo CP, Hariharan B (2021) Self-training for few-shot transfer across extreme ta skdifferences. In: 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. https://openreview.net/forum?id=O3Y56aqpChA

  17. Islam A, Chen C, Panda R, Karlinsky L, Radke RJ, Feris R (2021) A broad study on the transferability of visual representations with contrastive learning. In: IEEE/CVF International Conference on Computer Vision, ICCV, Montreal, QC, Canada, October 10-17, 2021, pp. 8825–8835. https://doi.org/10.1109/ICCV48922.2021.00872

  18. Lee K, Maji S, Ravichandran A, Soatto S (2019) Meta-learning with differentiable convex optimization. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR, Long Beach, CA, USA, June 16-20, 2019, pp. 10657–10665. https://doi.org/10.1109/CVPR.2019.01091 . http://openaccess.thecvf.com/content_CVPR_2019/html Lee_MetaLearning_With_Differentiable_Convex_Optimization_CVPR_2019_paper.html

  19. Vinyals O, Blundell C, Lillicrap T, Kavukcuoglu K, Wierstra D (2016) Matching networks for one shot learning. In: Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems, December 5-10, 2016, Barcelona, Spain, pp. 3630–3638. https://proceedings.neurips.cc/paper/2016/hash/90e1357833654983612fb05e3ec9148c-Abstract.html

  20. Snell J, Swersky K, Zemel RS (2017) Prototypical networks for few-shot learning. In: Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems, December 4-9, 2017, Long Beach, CA, USA, pp. 4077–4087. https://proceedings.neurips.cc/paper/2017/hash/cb8da6767461f2812ae4290eac7cbc42-Abstract.html

  21. Sung F, Yang Y, Zhang L, Xiang T, Torr PHS, Hospedales TM (2018) Learning to compare: Relation network for few-shot learning. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR, Salt Lake City, UT, USA, June 18-22, 2018, pp. 1199–1208. https://doi.org/10.1109/CVPR.2018.00131 . http://openaccess.thecvf.com/content_cvpr_2018/html/Sung_Learning_to_Compare_CVPR_2018_paper.html

  22. Finn C, Abbeel P, Levine S (2017) Model-agnostic meta-learning for fast adaptation of deep networks. In: Proceedings of the 34th International Conference on Machine Learning, ICML, Sydney, NSW, Australia, 6-11 August 2017. Proceedings of Machine Learning Research, vol. 70, pp. 1126–1135. http://proceedings.mlr.press/v70/finn17a.html

  23. Rajasegaran J, Khan S, Hayat M, Khan FS, Shah M (2021) Self-supervised knowledge distillation for few-shot learning. In: 32nd British Machine Vision Conference 2021, BMVC, Online, November 22-25, 2021, p. 179. https://www.bmvc2021-virtualconference.com/assets/papers/0820.pdf

  24. Yang S, Liu L, Xu M (2021) Free lunch for few-shot learning: Distribution calibration. In: 9th International Conference on Learning Representations, ICLR, Virtual Event, Austria, May 3-7, 2021. https://openreview.net/forum?id=JWOiYxMG92s

  25. Tian Y, Wang Y, Krishnan D, Tenenbaum JB, Isola P (2020) Rethinking few-shot image classification: A good embedding is all you need? In: Computer Vision - ECCV 2020 - 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XIV, vol. 12359, pp. 266–282. https://doi.org/10.1007/978-3-030-58568-6_16

  26. Chen Z, Wang C, Wu J, Deng C, Wang Y (2023) Deep convolutional transfer learning-based structural damage detection with domain adaptation. Applied Intelligence 53(5):5085–5099. https://doi.org/10.1007/s10489-022-03713-y

    Article  Google Scholar 

  27. Karimian M, Beigy H (2023) Concept drift handling: A domain adaptation perspective. Expert System with Applications 224:119946. https://doi.org/10.1016/j.eswa.2023.119946

    Article  Google Scholar 

  28. Kumar V, Patil H, Lal R, Chakraborty A (2023) Improving domain adaptation through class aware frequency transformation. International Journal of Computer Vision 131(11):2888–2907. https://doi.org/10.1007/s11263-023-01810-0

    Article  Google Scholar 

  29. Zhang J, Song J, Gao L, Shen H (2022) Free-lunch for cross-domain few-shot learning: Style-aware episodic training with robust contrastive learning. In: MM ’22: The 30th ACM International Conference on Multimedia, Lisboa, Portugal, October 10 - 14, 2022, pp. 2586–2594. https://doi.org/10.1145/3503161.3547835

  30. Li W, Liu X, Bilen H (2022) Cross-domain few-shot learning with task-specific adapters. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR, New Orleans, LA, USA, June 18-24, 2022, pp. 7151–7160. https://doi.org/10.1109/CVPR52688.2022.00702

  31. Guan J, Zhang M, Lu Z (2020) Large-scale cross-domain few-shot learning. In: 15th Asian Conference on Computer Vision, ACCV, Kyoto, Japan, November 30 - December 4, 2020, Revised Selected Papers, Part III. Lecture Notes in Computer Science, vol. 12624, pp. 474–491. https://doi.org/10.1007/978-3-030-69535-4_29

  32. Islam A, Chen CR, Panda R, Karlinsky L, Feris R, Radke RJ (2021) Dynamic distillation network for cross-domain few-shot recognition with unlabeled data. In: Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems, NeurIPS, December 6-14, 2021, Virtual, pp. 3584–3595. https://proceedings.neurips.cc/paper/2021/hash/1d6408264d31d453d556c60fe7d0459e-Abstract.html

  33. Oh J, Kim S, Ho N, Kim J, Song H, Yun S (2022) Understanding cross-domain few-shot learning based on domain similarity and few-shot difficulty. In: Conference on Neural Information Processing Systems, NeurIPS. http://papers.nips.cc/paper_files/paper/2022/hash/11b3ae28275461741026c46c0c786711-Abstract-Conference.html

  34. Chen X, Fan H, Girshick RB, He K (2020) Improved baselines with momentum contrastive learning. arXiv arxiv:2003.04297

  35. Chen T, Kornblith S, Norouzi M, Hinton GE (2020) A simple framework for contrastive learning of visual representations. In: Proceedings of the 37th International Conference on Machine Learning, ICML, 13-18 July 2020, Virtual Event. Proceedings of Machine Learning Research, vol. 119, pp. 1597–1607. http://proceedings.mlr.press/v119/chen20j.html

  36. Grill J, Strub F, Altché F, Tallec C, Richemond PH, Buchatskaya E, Doersch C, Pires BÁ, Guo Z, Azar MG, Piot B, Kavukcuoglu K, Munos R, Valko M (2020) Bootstrap your own latent - A new approach to self-supervised learning. In: Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, Virtual. https://proceedings.neurips.cc/paper/2020/hash/f3ada80d5c4ee70142b17b8192b2958e-Abstract.html

  37. Noroozi M, Favaro P (2016) Unsupervised learning of visual representations by solving jigsaw puzzles. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) 14th European Conference of Computer Vision, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part VI. Lecture Notes in Computer Science, vol. 9910, pp. 69–84. https://doi.org/10.1007/978-3-319-46466-4_5

  38. Borkowski P, Ciesielski K, Klopotek MA (2014) Unsupervised aggregation of categories for document labelling. In: Foundations of Intelligent Systems - 21st International Symposium, ISMIS, Roskilde, Denmark, June 25-27, 2014. Proceedings. Lecture Notes in Computer Science, vol. 8502, pp. 335–344. https://doi.org/10.1007/978-3-319-08326-1_34

  39. Wang D, Li T, Deng P, Liu J, Huang W, Zhang F (2023) A generalized deep learning algorithm based on NMF for multi-view clustering. IEEE Transactions on Big Data 9(1):328–340. https://doi.org/10.1109/TBDATA.2022.3163584

    Article  Google Scholar 

  40. Wang D, Li T, Deng P, Zhang F, Huang W, Zhang P, Liu J (2023) A generalized deep learning clustering algorithm based on non-negative matrix factorization. ACM Transactions on Knowledge Discovery from Data 17(7):99–19920. https://doi.org/10.1145/3584862

    Article  Google Scholar 

  41. Wang D, Li T, Huang W, Luo Z, Deng P, Zhang P, Ma M (2023) A multi-view clustering algorithm based on deep semi-nmf. Information Fusion 99:101884. https://doi.org/10.1016/J.INFFUS.2023.101884

    Article  Google Scholar 

  42. Wang D, Li T, Deng P, Luo Z, Zhang P, Liu K, Huang W (2024) Dnsrf: Deep network-based semi-nmf representation framework. ACM Transactions on Intelligent Systems and Technology. https://doi.org/10.1145/3670408

    Article  Google Scholar 

  43. Gidaris S, Singh P, Komodakis N (2018) Unsupervised representation learning by predicting image rotations. In: 6th International Conference on Learning Representations, ICLR, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings. https://openreview.net/forum?id=S1v4N2l0-

  44. Chen X, He K (2021) Exploring simple siamese representation learning. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR, Virtual, June 19-25, 2021, pp. 15750–15758. https://doi.org/10.1109/CVPR46437.2021.01549 . https://openaccess.thecvf.com/content/CVPR2021/html/Chen_Exploring_Simple_Siamese_Representation_Learning_CVPR_2021_paper.html

  45. Tian Y, Chen X, Ganguli S (2021) Understanding self-supervised learning dynamics without contrastive pairs. In: Proceedings of the 38th International Conference on Machine Learning, ICML, 18-24 July 2021, Virtual Event. Proceedings of Machine Learning Research, vol. 139, pp. 10268–10278. http://proceedings.mlr.press/v139/tian21a.html

  46. Zbontar J, Jing L, Misra I, LeCun Y, Deny S (2021) Barlow twins: Self-supervised learning via redundancy reduction. In: Proceedings of the 38th International Conference on Machine Learning, ICML, 18-24 July 2021, Virtual Event. Proceedings of Machine Learning Research, vol. 139, pp. 12310–12320. http://proceedings.mlr.press/v139/zbontar21a.html

  47. Ren M, Triantafillou E, Ravi S, Snell J, Swersky K, Tenenbaum JB, Larochelle H, Zemel RS (2018) Meta-learning for semi-supervised few-shot classification. In: 6th International Conference on Learning Representations, ICLR, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings. https://openreview.net/forum?id=HJcSzz-CZ

  48. Codella NCF, Rotemberg V, Tschandl P, Celebi ME, Dusza SW, Gutman DA, Helba B, Kalloo A, Liopyris K, Marchetti MA, Kittler H, Halpern A (2019) Skin lesion analysis toward melanoma detection 2018: A challenge hosted by the international skin imaging collaboration (ISIC). arXiv arxiv:1902.03368

  49. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR, Las Vegas, NV, USA, June 27-30, 2016, pp. 770–778. https://doi.org/10.1109/CVPR.2016.90

Download references

Acknowledgements

This work was supported by the Natural Science Basic Research Program of Shaanxi Province (Grant No.2021JQ-487) and the Key Laboratory of Manufacturing Equipment of Shaanxi Province (Grant No.JXZZZB-2022-02).

Author information

Authors and Affiliations

  1. Institute of Automation and Information Engineering, Xi’an University of Technology, No. 5 Jinhua South Road, Beilin District, Xi’an, 710048, Shaanxi, China

    Yuandong Bi, Hong Zhu, Jing Shi & Bin Song

Authors
  1. Yuandong Bi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hong Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jing Shi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Bin Song
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Yuandong Bi conceptualized and designed the algorithm, implemented the initial codebase, and prepared the original manuscript draft. Hong Zhu supervised the project, provided strategic direction in algorithm development and testing, and conducted a thorough review and final approval of the manuscript prior to submission. Jing Shi provided essential theoretical insights, contributed to algorithm improvements, and critically revised the manuscript for important intellectual content. Bin Song contributed to the development and fine-tuning of the algorithm.

Corresponding author

Correspondence to Hong Zhu.

Ethics declarations

Conflict of interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bi, Y., Zhu, H., Shi, J. et al. TsCANet: Three-stream contrastive adaptive network for cross-domain few-shot learning. J Supercomput 81, 139 (2025). https://doi.org/10.1007/s11227-024-06482-2

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11227-024-06482-2

Keywords

Search

Navigation