Two-branch weakly supervised semantic segmentation network based on web data enhancement
Article No.: 54, Pages 1 - 7
Abstract
As we all know, CAM [1] can only provide initial seeds of objects in weakly supervised segmentation. This paper proposes a two-branch semantic segmentation network that can transfer semantic knowledge, which can help CAM achieve more complete and accurate regions of objects. Thus, the network can be specifically divided into 3 steps: (1) Firstly, training the attention map to obtain the initial mask of the target dataset; (2)Secondly, Crawling images from the web to build a web dataset, use the trained attention map model to clean the web dataset, then transfer knowledge to target dataset by training group segmentation network; (3) Finally, Fusion of the initial target dataset masks and enhanced masks are used as pseudo-supervised training full convolutional networks. The backbone network are VGG16 and ResNet-101, image-level labels were used to obtain segmentation performances of Val 65.1% and Test 65.9% on the PASCAL VOC 2012 dataset.
References
[1]
Zhou, Bolei, "Learning deep features for discriminative localization." Proceedings of the IEEE conference on computer vision and pattern recognition. 2016.
[2]
Bearman A, Russakovsky O, Ferrari V, What's the Point: Semantic Segmentation with Point Supervision[C]// European Conference on Computer Vision. Springer, Cham, 2016.
[3]
Wang X, You S, Li X, Weakly-Supervised Semantic Segmentation by Iteratively Mining Common Object Features[C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, 2018.
[4]
Chen X, Gupta A. Webly supervised learning of convolutional networks[C]//Proceedings of the IEEE International Conference on Computer Vision. 2015: 1431-1439.
[5]
Krause J, Sapp B, Howard A, The unreasonable effectiveness of noisy data for fine-grained recognition[C]//European Conference on Computer Vision. Springer, Cham, 2016: 301-320.
[6]
Tong X, Tian X, Yi Y, Learning from massive noisy labeled data for image classification[C]// 2015 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 2015.
[7]
Chen X, Shrivastava A, Gupta A . NEIL: Extracting visual knowledge from web data[C]// 2013 IEEE International Conference on Computer Vision. IEEE, 2014.
[8]
Jin B, Segovia M, Susstrunk S . Webly Supervised Semantic Segmentation[C]// IEEE Conference on Computer Vision & Pattern Recognition. IEEE Computer Society, 2017:1705-1714.
[9]
Shen T, Lin G, Liu L, Weakly supervised semantic segmentation based on web image co-segmentation[J]. arXiv preprint arXiv:1705.09052, 2017.
[10]
Wei Y, Liang X, Chen Y, STC: A Simple to Complex Framework for Weakly-supervised Semantic Segmentation[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2017, 39(11):2314-2320.
[11]
Hong S, D Yeo, Kwak S, Weakly Supervised Semantic Segmentation Using Web-Crawled Videos[C]// IEEE Computer Society. IEEE Computer Society, 2017.
[12]
Shimoda W, Yanai K. Self-supervised difference detection for weakly-supervised semantic segmentation[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. 2019: 5208-5217.
[13]
Li X, Zhou T, Li J, Group-wise semantic mining for weakly supervised semantic segmentation[J]. arXiv preprint arXiv:2012.05007, 2020.
[14]
Kolesnikov A, Lampert C H. Seed, expand and constrain: Three principles for weakly-supervised image segmentation[C]//European conference on computer vision. Springer, Cham, 2016: 695-711.
[15]
Ahn J, Kwak S. Learning Pixel-level Semantic Affinity with Image-level Supervision for Weakly Supervised Semantic Segmentation[C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, 2018.
[16]
Hou Q, Jiang P T, Wei Y, Self-erasing network for integral object attention[J]. arXiv preprint arXiv:1810.09821, 2018.
[17]
Fan J, Zhang Z, Tan T, CIAN: Cross-Image Affinity Net for Weakly Supervised Semantic Segmentation[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2020, 34(7):10762-10769.
[18]
Lee J, Kim E, Lee S, FickleNet: Weakly and Semi-supervised Semantic Image Segmentation\\\\using Stochastic Inference[C]// IEEE. IEEE, 2019.
[19]
Wang Y, Zhang J, Kan M, Self-supervised equivariant attention mechanism for weakly supervised semantic segmentation[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020: 12275-12284.
[20]
Fan J, Zhang Z, Song C, Learning Integral Objects With Intra-Class Discriminator for Weakly-Supervised Semantic Segmentation[C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, 2020.
[21]
Zhang B, Xiao J, Wei Y, Reliability Does Matter: An End-to-End Weakly Supervised Semantic Segmentation Approach[C]// 2020:12765-12772.
[22]
K. Chen, P. P. K. Chan, T. Xiang, N. Kees and D. S. Yeung, "Class-Specific Affinity based Weakly Supervised Semantic Segmentation with Neutral Region Exploration," 2021 International Joint Conference on Neural Networks (IJCNN), Shenzhen, China, 2021, pp. 1-8.
[23]
M. Ergül and A. Alatan, "Depth is all you Need: Single-Stage Weakly Supervised Semantic Segmentation From Image-Level Supervision," 2022 IEEE International Conference on Image Processing (ICIP), Bordeaux, France, 2022, pp. 4233-4237.
[24]
Z. Zhang, "A Componentwise Approach to Weakly Supervised Semantic Segmentation Using Dual-Feedback Network," in IEEE Transactions on Neural Networks and Learning Systems.
[25]
Y. Li, J. Sun and Y. Li, "Weakly-Supervised Semantic Segmentation Network With Iterative dCRF," in IEEE Transactions on Intelligent Transportation Systems, vol. 23, no. 12, pp. 25419-25426, Dec. 2022.
[26]
Yan, X., Zheng, Z., Gao, Y. (2023). Improved Weakly Supervised Image Semantic Segmentation Method Based on SEC. In: Xiong, N., Li, M., Li, K., Xiao, Z., Liao, L., Wang, L. (eds) Advances in Natural Computation, Fuzzy Systems and Knowledge Discovery. ICNC-FSKD 2022. Lecture Notes on Data Engineering and Communications Technologies, vol 153. Springer, Cham. https://doi.org/10.1007/978-3-031-20738-9_71
Index Terms
- Two-branch weakly supervised semantic segmentation network based on web data enhancement
Index terms have been assigned to the content through auto-classification.
Recommendations
Weakly supervised semantic segmentation by iteratively refining optimal segmentation with deep cues guidance
AbstractWeakly supervised semantic segmentation under image-level label supervision has undergone impressive improvements over the past years. These approaches can significantly reduce human annotation efforts, although they remain inferior to fully ...
Weakly Supervised Random Forest for Multi-Label Image Clustering and Segmentation
ICMR '15: Proceedings of the 5th ACM on International Conference on Multimedia RetrievalClustering is a useful statistical tool in data mining and computer vision. Supervised information is introduced to improve the clustering performance. However, labeling each piece of data accurately is extremely expensive when the amount of data is ...
Weakly supervised semantic segmentation with segments and neighborhood classifiers
AbstractSemantic segmentation can provide basic semantic information for scene understanding, which has important theoretical research value and broad application prospects. Limited by the labeling cost and the scale of training data, weakly supervised ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
August 2023
378 pages
ISBN:9798400708701
DOI:10.1145/3627341
Copyright © 2023 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 15 December 2023
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
ICCVIT 2023
ICCVIT 2023: International Conference on Computer, Vision and Intelligent Technology
August 25 - 28, 2023
Chenzhou, China
Acceptance Rates
ICCVIT '23 Paper Acceptance Rate 54 of 142 submissions, 38%;
Overall Acceptance Rate 54 of 142 submissions, 38%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 16Total Downloads
- Downloads (Last 12 months)16
- Downloads (Last 6 weeks)1
Reflects downloads up to 12 Nov 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format