research-article

3D point cloud semantic segmentation toward large-scale unstructured agricultural scene classification

Authors:

Qian ZhangAuthors Info & Claims

Volume 190, Issue C

https://doi.org/10.1016/j.compag.2021.106445

Published: 01 November 2021 Publication History

Highlights

•

The 3D point cloud semantic segmentation network for large-scale unstructured agricultural scene was proposed.

•

The network of RandLA-Net semantic segmentation algorithm was improved.

•

Random sampling algorithm was replaced with farthest point sampling algorithm.

•

Our proposed algorithm shows applicability for 3D point cloud semantic segmentation of agricultural scenes.

Abstract

In recent years, with the development of computer vision, deep learning, and artificial intelligence technologies, the popularity of depth sensors and lidar has promoted the rapid development of three-dimensional (3D) point cloud semantic segmentation. The semantic segmentation of 3D point clouds for large-scale unstructured agricultural scenes is important for agricultural robots to perceive their surrounding environment, and for autonomous navigation and positioning and autonomous scene understanding. In this study, the problem of 3D point cloud semantic segmentation for large-scale unstructured agricultural scenes was studied. By improving the neural network structure of RandLA-Net, a deeper 3D point cloud semantic segmentation neural network model for large-scale unstructured agricultural scenes was built, and good experimental results were obtained. The local feature aggregation module in RandLA-Net was integrated and improved to achieve 3D point cloud semantic segmentation for large-scale unstructured agricultural scenes. To test the influence of the 3D point cloud sampling algorithm on the overall accuracy (OA) and mean intersection-over-union (mIoU) of semantic segmentation, the random sampling algorithm and farthest point sampling algorithm were used to build two models with the same neural network structure. The test results show that the sampling algorithm has little effect on the OA and mIoU of 3D point cloud semantic segmentation, and the final result depends mainly on the extraction of 3D point cloud features. In addition, two different Semantic3D datasets were used to test the effect of the datasets on the generalization ability of the model, and the results showed that the datasets had an important effect on the neural network model.

References

[1]

J. Behley, M. Garbade, A. Milioto, J. Quenzel, S. Behnke, C. Stachniss, J. Gall, Semantickitti: A dataset for semantic scene understanding of lidar sequences, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019, pp. 9297–9307.

[2]

A. Boulch, J. Guerry, B. Le Saux, N. Audebert, SnapNet: 3D point cloud semantic labeling with 2D deep segmentation networks, Computers Graphics 71 (2018) 189–198.

[3]

Chen, C., Fragonara, L.Z., Tsourdos, A., 2019. GAPNet: Graph attention based point neural network for exploiting local feature of point cloud. arXiv preprint arXiv:1905.08705.

[4]

Y.i. Chen, B. Zhang, J. Zhou, K. Wang, Real-time 3D unstructured environment reconstruction utilizing VR and Kinect-based immersive teleoperation for agricultural field robots, Comput. Electron. Agric. 175 (2020) 105579,.

[5]

S. Erfani, A. Jafari, A. Hajiahmad, Comparison of two data fusion methods for localization of wheeled mobile robot in farm conditions, Artificial Intelligence Agric. 1 (2019) 48–55.

[6]

J. Guerry, A. Boulch, B. Le Saux, J. Moras, A. Plyer, D. Filliat, Snapnet-r: Consistent 3d multi-view semantic labeling for robotics, in: Proceedings of the IEEE International Conference on Computer Vision Workshops, 2017, pp. 669–678.

[7]

Y. Guo, H. Wang, Q. Hu, H. Liu, L. Liu, M. Bennamoun, Deep learning for 3d point clouds: A survey, IEEE Trans. Pattern Anal. Mach. Intelligence (2020).

[8]

N. Guo, B. Zhang, J. Zhou, K. Zhan, S. Lai, Pose estimation and adaptable grasp configuration with point cloud registration and geometry understanding for fruit grasp planning, Comput. Electron. Agric. 179 (2020) 105818,.

[9]

Hackel, T., Savinov, N., Ladicky, L., Wegner, J.D., Schindler, K., Pollefeys, M., 2017. Semantic3d.net: A new large-scale point cloud classification benchmark. arXiv preprint arXiv:1704.03847.

[10]

Q. Hu, B. Yang, L. Xie, S. Rosa, Y. Guo, Z. Wang, A. Markham, Randla-net: Efficient semantic segmentation of large-scale point clouds, in: In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 11108–11117.

[11]

B.o. Jiang, J. He, S. Yang, H. Fu, T. Li, H. Song, D. He, Fusion of machine vision technology and AlexNet-CNNs deep learning network for the detection of postharvest apple pesticide residues, Artificial Intelligence Agric. 1 (2019) 1–8.

[12]

A. Komarichev, Z. Zhong, J. Hua, A-cnn: Annularly convolutional neural networks on point clouds, in: In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 7421–7430.

[13]

L. Landrieu, M. Simonovsky, Large-scale point cloud semantic segmentation with superpoint graphs, in: In Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 4558–4567.

[14]

Ma, Y., Guo, Y., Lei, Y., Lu, M., Zhang, J., 2018, August. 3DMAX-net: A multi-scale spatial contextual network for 3D point cloud semantic segmentation. In: 2018 24th International Conference on Pattern Recognition (ICPR). IEEE, pp. 1560–1566.

[15]

S. Mao, Y. Li, Y. Ma, B. Zhang, J. Zhou, Kai Wang, Automatic cucumber recognition algorithm for harvesting robots in the natural environment using deep learning and multi-feature fusion, Comput. Electron. Agric. 170 (2020) 105254,.

Digital Library

[16]

A.S. Mian, M. Bennamoun, R. Owens, Three-dimensional model-based object recognition and segmentation in cluttered scenes, IEEE Trans. Pattern Anal. Mach. Intell. 28 (10) (2006) 1584–1601.

Digital Library

[17]

M.G. Plessen, Coupling of crop assignment and vehicle routing for harvest planning in agriculture, Artificial Intelligence Agric. 2 (2019) 99–109.

[18]

Qi, C. R., Yi, L., Su, H., Guibas, L.J., 2017. Pointnet++: Deep hierarchical feature learning on point sets in a metric space. arXiv preprint arXiv:1706.02413.

[19]

C.R. Qi, H. Su, K. Mo, L.J. Guibas, Pointnet: Deep learning on point sets for 3d classification and segmentation, in: Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 652–660.

[20]

H. Su, S. Maji, E. Kalogerakis, E. Learned-Miller, Multi-view convolutional neural networks for 3d shape recognition, in: Proceedings of the IEEE international conference on computer vision, 2015, pp. 945–953.

Digital Library

[21]

G. Te, W. Hu, A. Zheng, Z. Guo, October). Rgcnn: Regularized graph cnn for point cloud segmentation, in: Proceedings of the 26th ACM international conference on Multimedia, 2018, pp. 746–754.

[22]

Y. Wang, M. Hu, Y. Zhou, Q. Li, N. Yao, G. Zhai, X.-P. Zhang, X. Yang, Unobtrusive and automatic classification of multiple people’s abnormal respiratory patterns in real time using deep neural network and depth camera, IEEE Internet Things J. 7 (9) (2020) 8559–8571.

[23]

Wang, Y., Yang, C., Hu, M., Zhang, J., Li, Q., Zhai, G., Zhang, X.P., 2021, June. Identification of deep breath while moving forward based on multiple body regions and graph signal analysis. In: ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, pp. 7958–7962.

[24]

Y. Xu, T. Fan, M. Xu, L. Zeng, Y. Qiao, Spidercnn: Deep learning on point sets with parameterized convolutional filters, in: Proceedings of the European Conference on Computer Vision (ECCV), 2018, pp. 87–102.

[25]

H. Yu, Z. Yang, L. Tan, Y. Wang, W. Sun, M. Sun, Y. Tang, Methods and datasets on semantic segmentation: a review, Neurocomputing 304 (2018) 82–103.

Digital Library

[26]

J.Y. Zhang, X.L. Zhao, Z. Chen, A review of semantic segmentation of point cloud based on deep learning, Las. Optoelect. Prog 57 (2020) 28–46.

[27]

Zhang, K., Hao, M., Wang, J., de Silva, C.W., Fu, C., 2019. Linked dynamic graph cnn: Learning on point cloud via linking hierarchical features. arXiv preprint arXiv:1904.10014.

Cited By

Liu QYang HWei JZhang YYang S(2024)FSDNETComputers and Electronics in Agriculture10.1016/j.compag.2024.109073222:COnline publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1016/j.compag.2024.109073
Bai YZhang BXu NZhou JShi JDiao Z(2023)Vision-based navigation and guidance for agricultural autonomous vehicles and robotsComputers and Electronics in Agriculture10.1016/j.compag.2022.107584205:COnline publication date: 1-Feb-2023
https://dl.acm.org/doi/10.1016/j.compag.2022.107584
Kang HWang X(2023)Semantic segmentation of fruits on multi-sensor fused data in natural orchardsComputers and Electronics in Agriculture10.1016/j.compag.2022.107569204:COnline publication date: 1-Jan-2023
https://dl.acm.org/doi/10.1016/j.compag.2022.107569
Show More Cited By

Index Terms

3D point cloud semantic segmentation toward large-scale unstructured agricultural scene classification
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
        Image segmentation
      2. Computer vision tasks
  2. Machine learning
    1. Learning paradigms
      1. Supervised learning
    2. Machine learning approaches

Index terms have been assigned to the content through auto-classification.

Recommendations

Towards 3D LiDAR-based semantic scene understanding of 3D point cloud sequences: The SemanticKITTI Dataset

A holistic semantic scene understanding exploiting all available sensor modalities is a core capability to master self-driving in complex everyday traffic. To this end, we present the SemanticKITTI dataset that provides point-wise semantic annotations of ...
Few-shot 3D Point Cloud Semantic Segmentation with Prototype Alignment
ICMLT '23: Proceedings of the 2023 8th International Conference on Machine Learning Technologies

Semantic Segmentation for 3D point clouds has made great progress in recent years. Most existing approaches for 3D point cloud segmentation are fully supervised, and they require a large number of well-annotated data for training. The training data is ...
Large-Scale Point Cloud Semantic Segmentation Network Based on Self-Learning Aggregation Sampling and Dual-Feature Attentive Pooling
AIPR '23: Proceedings of the 2023 6th International Conference on Artificial Intelligence and Pattern Recognition

Point cloud semantic segmentation is widely used in the fields of autonomous driving and robotics. In order to achieve more effective semantic segmentation, a point cloud semantic segmentation network based on self-learning aggregation sampling and dual-...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Computers and Electronics in Agriculture

Computers and Electronics in Agriculture Volume 190, Issue C

Nov 2021

810 pages

ISSN:0168-1699

Issue’s Table of Contents

Elsevier B.V.

Publisher

Elsevier Science Publishers B. V.

Netherlands

Publication History

Published: 01 November 2021

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 12 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Liu QYang HWei JZhang YYang S(2024)FSDNETComputers and Electronics in Agriculture10.1016/j.compag.2024.109073222:COnline publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1016/j.compag.2024.109073
Bai YZhang BXu NZhou JShi JDiao Z(2023)Vision-based navigation and guidance for agricultural autonomous vehicles and robotsComputers and Electronics in Agriculture10.1016/j.compag.2022.107584205:COnline publication date: 1-Feb-2023
https://dl.acm.org/doi/10.1016/j.compag.2022.107584
Kang HWang X(2023)Semantic segmentation of fruits on multi-sensor fused data in natural orchardsComputers and Electronics in Agriculture10.1016/j.compag.2022.107569204:COnline publication date: 1-Jan-2023
https://dl.acm.org/doi/10.1016/j.compag.2022.107569
Zhu TMa XGuan HWu XWang FYang CJiang Q(2023)A calculation method of phenotypic traits based on three-dimensional reconstruction of tomato canopyComputers and Electronics in Agriculture10.1016/j.compag.2022.107515204:COnline publication date: 1-Jan-2023
https://dl.acm.org/doi/10.1016/j.compag.2022.107515
Sharma VTripathi AMittal H(2022)Technological revolutions in smart farmingComputers and Electronics in Agriculture10.1016/j.compag.2022.107217201:COnline publication date: 1-Oct-2022
https://dl.acm.org/doi/10.1016/j.compag.2022.107217
Yu QYang HGao YMa XChen GWang X(2022)LFPNetComputers and Electronics in Agriculture10.1016/j.compag.2022.106691194:COnline publication date: 1-Mar-2022
https://dl.acm.org/doi/10.1016/j.compag.2022.106691

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents