research-article

Superpoint-guided Semi-supervised Semantic Segmentation of 3D Point Clouds

Authors:

Zhanyi HuAuthors Info & Claims

2022 International Conference on Robotics and Automation (ICRA)

Pages 9214 - 9220

https://doi.org/10.1109/ICRA46639.2022.9811904

Published: 23 May 2022 Publication History

Abstract

3D point cloud semantic segmentation is a challenging topic in the computer vision field. Most of the existing methods in literature require a large amount of fully labeled training data, but it is extremely time-consuming to obtain these training data by manually labeling massive point clouds. Addressing this problem, we propose a superpoint-guided semi-supervised segmentation network for 3D point clouds, which jointly utilizes a small portion of labeled scene point clouds and a large number of unlabeled point clouds for network training. The proposed network is iteratively updated with its predicted pseudo labels, where a superpoint generation module is introduced for extracting superpoints from 3D point clouds, and a pseudo-label optimization module is explored for automatically assigning pseudo labels to the unlabeled points under the constraint of the extracted superpoints. Additionally, there are some 3D points without pseudo-label supervision. We propose an edge prediction module to constrain features of edge points. A superpoint feature aggregation module and a superpoint feature consistency loss function are introduced to smooth superpoint features. Extensive experimental results on two 3D public datasets demonstrate that our method can achieve better performance than several state-of-the-art point cloud segmentation networks and several popular semi-supervised segmentation methods with few labeled scenes.

References

[1]

C. R. Qi, H. Su, K. Mo, and L. J. Guibas, “Pointnet: Deep learning on point sets for 3d classification and segmentation,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017, pp. 652–660.

[2]

C. R. Qi, L. Yi, H. Su, and L. J. Guibas, “Pointnet++: Deep hierarchi-cal feature learning on point sets in a metric space,” in Proceedings of the Conference on Neural Information Processing Systems (NeurIPS), 2017, pp. 5099–5108.

[3]

B. Graham, M. Engelcke, and L. van der Maaten, “3d semantic segmentation with submanifold sparse convolutional networks,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018, pp. 9224–9232.

[4]

Y. Li, R. Bu, M. Sun, W. Wu, X. Di, and B. Chen, “Pointcnn: Convo-lution on x-transformed points,” in Proceedings of the Conference on Neural Information Processing Systems (NeurIPS), 2018, pp. 820–830.

[5]

L. Landrieu and M. Simonovsky, “Large-scale point cloud semantic segmentation with superpoint graphs,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018, pp. 4558–4567.

[6]

Y. Wang, Y. Sun, Z. Liu, S. E. Sarma, M. M. Bronstein, and J. M. Solomon, “Dynamic graph cnn for learning on point clouds,” ACM Transaction on Graphics (TOG), vol. 38, pp. 1–12, 2019.

Digital Library

[7]

H. Zhao, L. Jiang, C.-W. Fu, and J. Jia, “Pointweb: Enhancing local neighborhood features for point cloud processing,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019, pp. 5565–5573.

[8]

L. Landrieu and M. Boussaha, “Point cloud oversegmentation with graph-structured deep metric learning,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019, pp. 7432–7441.

[9]

Q. Hu, B. Yang, L. Xie, S. Rosa, Y. Guo, Z. Wang, N. Trigoni, and A. Markham, “Randla-net: Efficient semantic segmentation of large-scale point clouds,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020, pp. 11105–11114.

[10]

Q. Xu, X. Sun, C.-Y. Wu, P. Wang, and U. Neumann, “Grid-gcn for fast and scalable point cloud learning,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020, pp. 5661–5670.

[11]

S. Deng, B. Liu, Q. Dong, and Z. Hu, “Rotation transformation network: Learning view-invariant point cloud for classification and segmentation,” in Proceedings of the IEEE International Conference on Multimedia and Expo (ICME), 2021, pp. 1–6.

[12]

S. Fan, Q. Dong, F. Zhu, Y. Lv, P. Ye, and F.-Y. Wang, “Scf-net: Learning spatial contextual features for large-scale point cloud segmentation,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021, pp. 14504–14513.

[13]

S. Deng and Q. Dong, “Ga-net: Global attention network for point cloud semantic segmentation,” IEEE Signal Processing Letters (SPL), vol. 28, pp. 1300–1304, 2021.

[14]

Y. Wang, S. Asafi, O. van Kaick, H. Zhang, D. Cohen-Or, and B. Chen, “Active co-analysis of a set of shapes,” ACM Transactions on Graphics (TOG), vol. 31, pp. 1–10, 2012.

Digital Library

[15]

X. Xu and G. H. Lee, “Weakly supervised semantic point cloud segmentation: Towards 10x fewer labels,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020, pp. 13706–13715.

[16]

J. Mei, B. Gao, D. Xu, W. Yao, X. Zhao, and H. Zhao, “Semantic segmentation of 3d lidar data in dynamic scene using semi-supervised learning,” IEEE Transactions on Intelligent Transportation Systems (TITS), vol. 21, pp. 2496–2509, 2020.

[17]

H. Li, Z. Sun, Y. Wu, and Y. Song, “Semi-supervised point cloud segmentation using self-training with label confidence prediction,” Neurocomputing, vol. 437, pp. 227–237, 2021.

[18]

M. Cheng, L. Hui, J. Xie, and J. Yang, “Sspc-net: Semi-supervised semantic 3d point cloud segmentation network,” in Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), 2021, pp. 1140–1147.

[19]

T.-H. Wu, Y.-C. Liu, Y.-K. Huang, H.-Y. Lee, H.-T. Su, P.-C. Huang, and W. H. Hsu, “Redal: Region-based and diversity-aware active learning for point cloud semantic segmentation,” in Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2021, pp. 15510–15519.

[20]

X. Shi, X. Xu, K. Chen, L. Cai, C. S. Foo, and K. Jia, “Label-efficient point cloud semantic segmentation: An active learning approach,” arXiv preprint:, 2021.

[21]

Z. Liu, X. Qi, and C.-W. Fu, “One thing one click: A self-training approach for weakly supervised 3d semantic segmentation,” in Pro-ceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021, pp. 1726–1736.

[22]

J. Hou, B. Graham, M. Nießner, and S. Xie, “Exploring data-efficient 3d scene understanding with contrastive scene contexts,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021, pp. 15587–15597.

[23]

Q. Hu, B. Yang, G. Fang, Y. Guo, A. Leonardis, N. Trigoni, and A. Markham, “Sqn: Weakly-supervised semantic segmentation of large-scale 3d point clouds with 1000x fewer labels,” arXiv preprint:, 2021.

[24]

Y. Zhang, Y. Qu, Y. Xie, Z. Li, S. Zheng, and C. Li, “Perturbed self-distillation: Weakly supervised large-scale point cloud semantic segmentation,” in Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2021, pp. 15520–15528.

[25]

S. Laine and T. Aila, “Temporal ensembling for semi-supervised learning,” in Proceedings of the International Conference on Learning Representations (ICLR), 2017, pp. 1–13.

[26]

A. Tarvainen and H. Valpola, “Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results,” in Proceedings of the Conference on Neural Infor-mation Processing Systems (NeurIPS), 2017, p. 1195–1204.

[27]

N. Souly, C. Spampinato, and M. Shah, “Semi supervised semantic segmentation using generative adversarial network,” in Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2017, pp. 5689–5697.

[28]

X. Luo, J. Chen, T. Song, and G. Wang, “Semi-supervised medical image segmentation through dual-task consistency,” in Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), 2021, pp. 8801–8809.

[29]

M. Cheng, L. Hui, J. Xie, J. Yang, and H. Kong, “Cascaded non-local neural network for point cloud semantic segmentation,” in Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2020, pp. 8447–8452.

[30]

R. Adams and L. Bischof, “Seeded region growing,” IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), vol. 16, pp. 641–647, 1994.

Digital Library

[31]

O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional networks for biomedical image segmentation,” in International Con-ference on Medical Image Computing and Computer-Assisted Inter-vention (MMICCAI), 2015, pp. 234–241.

[32]

A. L. Maas, A. Y. Hannun, and A. Y. Ng, “Rectifier nonlinearities improve neural network acoustic models,” in Proceedings of the International Conference on Machine Learning (ICML), 2013, pp. 1–6.

[33]

D.-H. Lee, “Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks,” in Proceedings of the International Conference on Machine Learning Workshop (ICMLW), 2013, pp. 896–901.

[34]

I. Armeni, O. Sener, A. R. Zamir, H. Jiang, I. Brilakis, M. Fischer, and S. Savarese, “3d semantic parsing of large-scale indoor spaces,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 1534–1543.

[35]

A. Dai, A. X. Chang, M. Savva, M. Halber, T. Funkhouser, and M. Nießner, “Scannet: Richly-annotated 3d reconstructions of indoor scenes,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017, pp. 2432–2443.

[36]

S. Ushakov, “Region growing segmentation,” https://pcl.readthedocs.io/projects/tutorials/en/latest/region_growing_segmentation.html?highlight=region%20growing

[37]

S. Ushakov, “Color-based region growing segmentation,” https://pcl.readthedocs.io/projects/tutorials/en/latest/region_growing_rgb.segmentation.html?highlight=region%20growing

Cited By

Qiu BZhou YDai LWang BLi JDong ZWen CMa ZYang B(2024)WHU-Railway3D: A Diverse Dataset and Benchmark for Railway Point Cloud Semantic SegmentationIEEE Transactions on Intelligent Transportation Systems10.1109/TITS.2024.346954625:12(20900-20916)Online publication date: 14-Oct-2024
https://dl.acm.org/doi/10.1109/TITS.2024.3469546
Xie BLi SGuo QLiu CCheng XOh ANaumann TGloberson ASaenko KHardt MLevine S(2023)AnnotatorProceedings of the 37th International Conference on Neural Information Processing Systems10.5555/3666122.3668224(48444-48458)Online publication date: 10-Dec-2023
https://dl.acm.org/doi/10.5555/3666122.3668224

Recommendations

Semi- and Weakly- Supervised Semantic Segmentation with Deep Convolutional Neural Networks
MM '15: Proceedings of the 23rd ACM international conference on Multimedia

Successful semantic segmentation methods typically rely on the training datasets containing a large number of pixel-wise labeled images. To alleviate the dependence on such a fully annotated training dataset, in this paper, we propose a semi- and weakly-...
Bayesian Self-training for Semi-supervised 3D Segmentation
Computer Vision – ECCV 2024
Abstract
3D segmentation is a core problem in computer vision and, similarly to many other dense prediction tasks, it requires large amounts of annotated data for adequate training. However, densely labeling 3D point clouds to employ fully-supervised ...
Dense Supervision Propagation for Weakly Supervised Semantic Segmentation on 3D Point Clouds
Semantic segmentation on 3D point clouds is an important task for 3D scene understanding. While dense labeling on 3D data is expensive and time-consuming, only a few works address weakly supervised semantic point cloud segmentation methods to relieve the ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Guide Proceedings

2022 International Conference on Robotics and Automation (ICRA)

May 2022

6634 pages

Copyright © 2022.

Publisher

IEEE Press

Publication History

Published: 23 May 2022

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Qiu BZhou YDai LWang BLi JDong ZWen CMa ZYang B(2024)WHU-Railway3D: A Diverse Dataset and Benchmark for Railway Point Cloud Semantic SegmentationIEEE Transactions on Intelligent Transportation Systems10.1109/TITS.2024.346954625:12(20900-20916)Online publication date: 14-Oct-2024
https://dl.acm.org/doi/10.1109/TITS.2024.3469546
Xie BLi SGuo QLiu CCheng XOh ANaumann TGloberson ASaenko KHardt MLevine S(2023)AnnotatorProceedings of the 37th International Conference on Neural Information Processing Systems10.5555/3666122.3668224(48444-48458)Online publication date: 10-Dec-2023
https://dl.acm.org/doi/10.5555/3666122.3668224

View Options

View options

Figures

Tables

Media

View Table of Conten