research-article

Few-shot domain adaptation for semantic segmentation

Authors:

Dongyu ZhangAuthors Info & Claims

ACM TURC '19: Proceedings of the ACM Turing Celebration Conference - China

Article No.: 96, Pages 1 - 6

https://doi.org/10.1145/3321408.3322629

Published: 17 May 2019 Publication History

Abstract

Unsupervised domain adaptation (UDA) has attracted increasing attention in recent years because it can liberate the labor force of annotating data. However, in many cases, the target data is scarce because it is difficult to obtain, at this time, the supervised domain adaptation (SDA) becomes attractive. In this work, we propose a novel few-shot supervised domain adaptation framework for semantic segmentation. The main idea is to exploit adversarial learning to align the features extracted from networks. To address the challenge of scarce data, we propose a pairing method of creating pairs using source data and target data. We design our framework as a two-stage structure to enhance the adaptation of the low-level features. In order to ensure the stable and effective training, we employ the spectral normalization in the discriminators and propose an alternately training strategy for the whole framework. Our proposed framework can work well even when there is only one sample per category. We evaluate our proposed method on a challenging synthetic dataset to real-world dataset adaptation where the results demonstrate the effectiveness of our method.

References

[1]

Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos, Kevin Murphy, and Alan L Yuille. 2014. Semantic image segmentation with deep convolutional nets and fully connected crfs. arXiv preprint arXiv:1412.7062 (2014).

[2]

Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos, Kevin Murphy, and Alan L Yuille. 2018. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE transactions on pattern analysis and machine intelligence 40, 4 (2018), 834--848.

[3]

Yi-Hsin Chen, Wei-Yu Chen, Yu-Ting Chen, Bo-Cheng Tsai, Yu-Chiang Frank Wang, and Min Sun. 2017. No more discrimination: Cross city adaptation of road scene segmenters. In Proceedings of the IEEE International Conference on Computer Vision. 1992--2001.

[4]

Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele. 2016. The cityscapes dataset for semantic urban scene understanding. In Proceedings of the IEEE conference on computer vision and pattern recognition. 3213--3223.

[5]

Jifeng Dai, Kaiming He, and Jian Sun. 2015. Boxsup: Exploiting bounding boxes to supervise convolutional networks for semantic segmentation. In Proceedings of the IEEE International Conference on Computer Vision. 1635--1643.

Digital Library

[6]

Hal Daumé III. 2009. Frustratingly easy domain adaptation. arXiv preprint arXiv:0907.1815 (2009).

[7]

Yaroslav Ganin and Victor Lempitsky. 2014. Unsupervised domain adaptation by backpropagation. arXiv preprint arXiv:1409.7495 (2014).

Digital Library

[8]

Boqing Gong, Yuan Shi, Fei Sha, and Kristen Grauman. 2012. Geodesic flow kernel for unsupervised domain adaptation. In Computer Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on. IEEE, 2066--2073.

Digital Library

[9]

Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative adversarial nets. In Advances in neural information processing systems. 2672--2680.

Digital Library

[10]

Raghuraman Gopalan, Ruonan Li, and Rama Chellappa. 2011. Domain adaptation for object recognition: An unsupervised approach. In Computer Vision (ICCV), 2011 IEEE International Conference on. IEEE, 999--1006.

Digital Library

[11]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 770--778.

[12]

Judy Hoffman, Eric Tzeng, Taesung Park, Jun-Yan Zhu, Phillip Isola, Kate Saenko, Alexei A Efros, and Trevor Darrell. 2017. Cycada: Cycle-consistent adversarial domain adaptation. arXiv preprint arXiv:1711.03213 (2017).

[13]

Judy Hoffman, Dequan Wang, Fisher Yu, and Trevor Darrell. 2016. Fcns in the wild: Pixel-level adversarial and constraint-based adaptation. arXiv preprint arXiv:1612.02649 (2016).

[14]

Sergey Ioffe and Christian Szegedy. 2015. Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167 (2015).

Digital Library

[15]

Diederik Kingma and Jimmy Ba. 2014. Adam: A Method for Stochastic Optimization. Computer Science (2014).

[16]

Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. 2012. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems. 1097--1105.

Digital Library

[17]

Guosheng Lin, Chunhua Shen, Anton Van Den Hengel, and Ian Reid. 2016. Efficient piecewise training of deep structured models for semantic segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 3194--3203.

[18]

Jonathan Long, Evan Shelhamer, and Trevor Darrell. 2015. Fully convolutional networks for semantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition. 3431--3440.

[19]

Mingsheng Long, Yue Cao, Jianmin Wang, and Michael I Jordan. 2015. Learning transferable features with deep adaptation networks. arXiv preprint arXiv:1502.02791 (2015).

[20]

Mingsheng Long, Han Zhu, Jianmin Wang, and Michael I Jordan. 2016. Unsupervised domain adaptation with residual transfer networks. In Advances in Neural Information Processing Systems. 136--144.

Digital Library

[21]

Takeru Miyato, Toshiki Kataoka, Masanori Koyama, and Yuichi Yoshida. 2018. Spectral normalization for generative adversarial networks. arXiv preprint arXiv:1802.05957 (2018).

[22]

Saeid Motiian, Quinn Jones, Seyed Iranmanesh, and Gianfranco Doretto. 2017. Few-shot adversarial domain adaptation. In Advances in Neural Information Processing Systems. 6670--6680.

Digital Library

[23]

Saeid Motiian, Marco Piccirilli, Donald A Adjeroh, and Gianfranco Doretto. 2017. Unified deep supervised domain adaptation and generalization. In The IEEE International Conference on Computer Vision (ICCV), Vol. 2. 3.

[24]

Xinlei Pan, Yurong You, Ziyan Wang, and Cewu Lu. 2017. Virtual to real reinforcement learning for autonomous driving. arXiv preprint arXiv:1704.03952 (2017).

[25]

Deepak Pathak, Philipp Krahenbuhl, and Trevor Darrell. 2015. Constrained convolutional neural networks for weakly supervised segmentation. In Proceedings of the IEEE international conference on computer vision. 1796--1804.

Digital Library

[26]

German Ros, Laura Sellart, Joanna Materzynska, David Vazquez, and Antonio M Lopez. 2016. The synthia dataset: A large collection of synthetic images for semantic segmentation of urban scenes. In Proceedings of the IEEE conference on computer vision and pattern recognition. 3234--3243.

[27]

Andrei A Rusu, Matej Vecerik, Thomas Rothörl, Nicolas Heess, Razvan Pascanu, and Raia Hadsell. 2016. Sim-to-real robot learning from pixels with progressive nets. arXiv preprint arXiv:1610.04286 (2016).

[28]

Marvin Teichmann, Michael Weber, Marius Zoellner, Roberto Cipolla, and Raquel Urtasun. 2018. Multinet: Real-time joint semantic reasoning for autonomous driving. In 2018 IEEE Intelligent Vehicles Symposium (IV). IEEE, 1013--1020.

Digital Library

[29]

Yi-Hsuan Tsai, Wei-Chih Hung, Samuel Schulter, Kihyuk Sohn, Ming-Hsuan Yang, and Manmohan Chandraker. 2018. Learning to adapt structured output space for semantic segmentation. arXiv preprint arXiv:1802.10349 (2018).

[30]

Eric Tzeng, Judy Hoffman, Kate Saenko, and Trevor Darrell. 2017. Adversarial discriminative domain adaptation. In Computer Vision and Pattern Recognition (CVPR), Vol. 1. 4.

[31]

Zhang Yang, Philip David, and Boqing Gong. 2017. Curriculum Domain Adaptation for Semantic Segmentation of Urban Scenes. (2017).

[32]

Hengshuang Zhao, Xiaojuan Qi, Xiaoyong Shen, Jianping Shi, and Jiaya Jia. 2018. Icnet for real-time semantic segmentation on high-resolution images. In Proceedings of the European Conference on Computer Vision (ECCV). 405--420.

Digital Library

[33]

Hengshuang Zhao, Jianping Shi, Xiaojuan Qi, Xiaogang Wang, and Jiaya Jia. 2017. Pyramid scene parsing network. In IEEE Conf. on Computer Vision and Pattern Recognition (CVPR). 2881--2890.

Cited By

Wang WDuan LWang YFan JZhang Z(2023)MMT: Cross Domain Few-Shot Learning via Meta-Memory TransferIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2023.330635245:12(15018-15035)Online publication date: Dec-2023
https://doi.org/10.1109/TPAMI.2023.3306352
Zhu STian YZhou FBai KSong X(2022)COCM: Co-Occurrence-Based Consistency Matching in Domain-Adaptive SegmentationMathematics10.3390/math1023446810:23(4468)Online publication date: 26-Nov-2022
https://doi.org/10.3390/math10234468
Wang WDuan LWang YEn QFan JZhang Z(2022)Remember the Difference: Cross-Domain Few-Shot Semantic Segmentation via Meta-Memory Transfer2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52688.2022.00693(7055-7064)Online publication date: Jun-2022
https://doi.org/10.1109/CVPR52688.2022.00693
Show More Cited By

Index Terms

Few-shot domain adaptation for semantic segmentation
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
        Image segmentation
  2. Machine learning
    1. Learning paradigms
      1. Multi-task learning
        Learning under covariate shift
    2. Machine learning approaches
      1. Neural networks

Recommendations

Few-Shot Adaptation for Multimedia Semantic Indexing
MM '18: Proceedings of the 26th ACM international conference on Multimedia

We propose a few-shot adaptation framework, which bridges zero-shot learning and supervised many-shot learning, for semantic indexing of image and video data. Few-shot adaptation provides robust parameter estimation with few training examples, by ...
Cross-Domain Few-Shot Semantic Segmentation
Computer Vision – ECCV 2022
Abstract
Few-shot semantic segmentation aims at learning to segment a novel object class with only a few annotated examples. Most existing methods consider a setting where base classes are sampled from the same domain as the novel classes. However, in many ...
An unsupervised domain adaptation brain CT segmentation method across image modalities and diseases
Abstract
Computed tomography (CT) is the primary diagnostic tool for brain diseases. To determine the appropriate treatment plan, it is necessary to ascertain the patient’s bleeding volume. Automatic segmentation algorithms for hemorrhagic ...
Highlights
- Solve unsupervised segmentation problem in a novel domain adaptation way.
- ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

ACM TURC '19: Proceedings of the ACM Turing Celebration Conference - China

May 2019

963 pages

ISBN:9781450371582

DOI:10.1145/3321408

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 May 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

ACM TURC 2019

ACM TURC 2019: ACM Turing Celebration Conference - China

May 17 - 19, 2019

Chengdu, China

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
351
Total Downloads

Downloads (Last 12 months)15
Downloads (Last 6 weeks)2

Reflects downloads up to 14 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Wang WDuan LWang YFan JZhang Z(2023)MMT: Cross Domain Few-Shot Learning via Meta-Memory TransferIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2023.330635245:12(15018-15035)Online publication date: Dec-2023
https://doi.org/10.1109/TPAMI.2023.3306352
Zhu STian YZhou FBai KSong X(2022)COCM: Co-Occurrence-Based Consistency Matching in Domain-Adaptive SegmentationMathematics10.3390/math1023446810:23(4468)Online publication date: 26-Nov-2022
https://doi.org/10.3390/math10234468
Wang WDuan LWang YEn QFan JZhang Z(2022)Remember the Difference: Cross-Domain Few-Shot Semantic Segmentation via Meta-Memory Transfer2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52688.2022.00693(7055-7064)Online publication date: Jun-2022
https://doi.org/10.1109/CVPR52688.2022.00693
Liang PWu PZiyin LMorency LSalakhutdinov RShen HZhuang YSmith JYang YCesar PMetze FPrabhakaran B(2021)Cross-Modal Generalization: Learning in Low Resource Modalities via Meta-AlignmentProceedings of the 29th ACM International Conference on Multimedia10.1145/3474085.3475247(2680-2689)Online publication date: 17-Oct-2021
https://dl.acm.org/doi/10.1145/3474085.3475247
Li JWang WLiu XLiao L(2021)Analysis of the nonperfused volume ratio of adenomyosis from MRI images based on fewshot learningPhysics in Medicine & Biology10.1088/1361-6560/abd66b66:4(045019)Online publication date: 8-Feb-2021
https://doi.org/10.1088/1361-6560/abd66b
Choi JSharma GSchulter SHuang J(2020)Shuffle and Attend: Video Domain AdaptationComputer Vision – ECCV 202010.1007/978-3-030-58610-2_40(678-695)Online publication date: 7-Oct-2020
https://doi.org/10.1007/978-3-030-58610-2_40
Mei KZhu CZou JZhang S(2020)Instance Adaptive Self-training for Unsupervised Domain AdaptationComputer Vision – ECCV 202010.1007/978-3-030-58574-7_25(415-430)Online publication date: 13-Nov-2020
https://doi.org/10.1007/978-3-030-58574-7_25
Li GKang GLiu WWei YYang Y(2020)Content-Consistent Matching for Domain Adaptive Semantic SegmentationComputer Vision – ECCV 202010.1007/978-3-030-58568-6_26(440-456)Online publication date: 13-Nov-2020
https://doi.org/10.1007/978-3-030-58568-6_26

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents