Nothing Special   »   [go: up one dir, main page]

skip to main content
research-article

Semi-automated dataset creation for semantic and instance segmentation of industrial point clouds.

Published: 14 March 2024 Publication History

Abstract

The current practice for creating as-built geometric Digital Twins (gDTs) of industrial facilities is both labour-intensive and error-prone. In aged industries it typically involves manually crafting a CAD or BIM model from a point cloud collected using terrestrial laser scanners. Recent advances within deep learning (DL) offer the possibility to automate semantic and instance segmentation of point clouds, contributing to a more efficient modelling process. DL networks, however, are data-intensive, requiring large domain-specific datasets. Producing labelled point cloud datasets involves considerable manual labour, and in the industrial domain no open-source instance segmentation dataset exists. We propose a semi-automatic workflow leveraging object descriptions contained in existing gDTs to efficiently create semantic- and instance-labelled point cloud datasets. To prove the efficiency of our workflow, we apply it to two separate areas of a gas processing plant covering a total of 40 000 m 2. We record the effort needed to process one of the areas, labelling a total of 260 million points in 70 h. When benchmarking on a state-of-the-art 3D instance segmentation network, the additional data from the 70-hour effort raises mIoU from 24.4% to 44.4%, AP from 19.7% to 52.5% and RC from 45.9% to 76.7% respectively.

Highlights

A workflow for creating semantic- and -instance labelled real point cloud datasets.
A benchmark of the SoftGroup DL Network in the industrial domain.
Indoor 3D instance segmentation networks are applicable in the industrial domain.

References

[1]
Agapaki E., Brilakis I., CLOI-NET: Class segmentation of industrial facilities’ point cloud datasets, Adv. Eng. Inform. 45 (2020),. URL: https://www.sciencedirect.com/science/article/pii/S1474034620300902.
[2]
Agapaki E., Brilakis I.K., Instance segmentation of industrial point cloud data, 2020, CoRR abs/2012.14253, arXiv:2012.14253, URL: https://arxiv.org/abs/2012.14253.
[3]
Agapaki, E., Glyn-Davies, A., Mandoki, S., Brilakis, I., 2019. CLOI: A shape classification benchmark dataset for industrial facilities. In: Computing in Civil Engineering 2019. pp. 66–73. https://doi.org/10.1061/9780784482445.009, arXiv:https://ascelibrary.org/doi/pdf/10.1061/9780784482445.009, URL: https://ascelibrary.org/doi/abs/10.1061/9780784482445.009.
[4]
Bentley J.L., Multidimensional binary search trees used for associative searching, Commun. ACM 18 (9) (1975) 509–517,.
[5]
Birkeland A.A., Streamlining the Creation of Industrial Digital Twins with 3D Semantic and Instance Segmentation, Norwegian University of Science and Technology, 2023, Unpublished master’s thesis.
[6]
Chen S., Fang J., Zhang Q., Liu W., Wang X., Hierarchical aggregation for 3D instance segmentation, 2021, https://doi.org/10.48550/ARXIV.2108.02350, URL: https://arxiv.org/abs/2108.02350.
[7]
Czerniawski, T., Leite, F., 2019. Semantic segmentation of building point clouds using deep learning: A method for creating training data using BIM to point cloud label transfer. In: Computing in Civil Engineering 2019. pp. 410–416. arXiv:https://ascelibrary.org/doi/pdf/10.1061/9780784482421.052, URL: https://ascelibrary.org/doi/abs/10.1061/9780784482421.052, https://doi.org/10.1061/9780784482421.052.
[8]
De Geyter S., Bassier M., Vergauwen M., Automated Training Data Creation for Semantic Segmentation of 3d Point Clouds, in: The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, vol. XLVI-5/W1-2022, 2022, pp. 59–67,.
[9]
de las Mercedes Gonzalez Stefanelli M.J., Jain A.R., Jalui S.K., Agapaki E., Synthetic point cloud generation for class segmentation applications, in: Proceedings of the 2022 European Conference on Computing in Construction, in: Computing in Construction, vol. 3, University of Turin, Rhodes, Greece, 2022,. URL: https://ec-3.org/publications/conference/paper/?id=EC32022_197.
[10]
Graham B., Engelcke M., van der Maaten L., 3D semantic segmentation with submanifold sparse convolutional networks, 2017, CoRR abs/1711.10275, arXiv:1711.10275, URL: http://arxiv.org/abs/1711.10275.
[11]
Jiang L., Zhao H., Shi S., Liu S., Fu C.-W., Jia J., PointGroup: Dual-set point grouping for 3D instance segmentation, 2020, https://doi.org/10.48550/ARXIV.2004.01658, URL: https://arxiv.org/abs/2004.01658.
[12]
Kaufmann F., Glock C., Tschickardt T., ScaleBIM: Introducing a scalable modular framework to transfer point clouds into semantically rich building information models, in: Proceedings of the 2022 European Conference on Computing in Construction, in: Computing in Construction, vol. 3, University of Turin, Rhodes, Greece, 2022,. URL: https://ec-3.org/publications/conference/paper/?id=EC32022_194.
[13]
Kolodiazhnyi M., Rukhovich D., Vorontsova A., Konushin A., Top-down beats bottom-up in 3D instance segmentation, 2023, arXiv:2302.02871.
[14]
Ma J.W., Czerniawski T., Leite F., Semantic segmentation of point clouds of building interiors with deep learning: Augmenting training datasets with synthetic bim-based point clouds, Autom. Constr. 113 (2020),. URL: https://www.sciencedirect.com/science/article/pii/S0926580519311884.
[15]
Ma J.W., Han B., Leite F., An automated framework for generating synthetic point clouds from as-built BIM with semantic annotation for scan-to-BIM, in: 2021 Winter Simulation Conference (WSC), 2021, pp. 1–10,.
[16]
Ngo T.D., Hua B.-S., Nguyen K., ISBNet: a 3D point cloud instance segmentation network with instance-aware sampling and box-aware dynamic convolution, 2023, arXiv:2303.00246.
[17]
Noichl F., Braun A., Borrmann A., BIM-to-scan for scan-to-BIM: Generating realistic synthetic ground truth point clouds based on industrial 3D models, in: Proceedings of the 2021 European Conference on Computing in Construction, in: Computing in Construction, vol. 2, ETH, Online Conference, 2021, pp. 164–172,. URL: https://ec-3.org/publications/conference/paper/?id=EC32021_166.
[18]
Park J., Kim J., Lee D., Jeong K., Lee J., Kim H., Hong T., Deep learning–based automation of scan-to-BIM with modeling objects from occluded point clouds, J. Manage. Eng. 38 (4) (2022),. arXiv:https://ascelibrary.org/doi/pdf/10.1061/(ASCE)ME.1943-5479.0001055, URL: https://ascelibrary.org/doi/abs/10.1061/(ASCE)ME.1943-5479.0001055.
[19]
Park J., Kim J., Lee D., Jeong K., Lee J., Kim H., Hong T., Deep learning–based automation of scan-to-BIM with modeling objects from occluded point clouds, J. Manage. Eng. 38 (4) (2022),.
[20]
Qi C.R., Su H., Mo K., Guibas L.J., PointNet: Deep learning on point sets for 3D classification and segmentation, 2017, arXiv:1612.00593.
[21]
Qi C.R., Yi L., Su H., Guibas L.J., PointNet++: Deep hierarchical feature learning on point sets in a metric space, 2017, CoRR abs/1706.02413, arXiv:1706.02413, URL: http://arxiv.org/abs/1706.02413.
[22]
Schult J., Engelmann F., Hermans A., Litany O., Tang S., Leibe B., Mask3D for 3D semantic instance segmentation, 2022, https://doi.org/10.48550/ARXIV.2210.03105, URL: https://arxiv.org/abs/2210.03105.
[23]
Singh M., Srivastava R., Fuenmayor E., Kuts V., Qiao Y., Murray N., Devine D., Applications of digital twin across industries: A review, Appl. Sci. 12 (11) (2022),. URL: https://www.mdpi.com/2076-3417/12/11/5727.
[24]
Sun J., Qing C., Tan J., Xu X., Superpoint transformer for 3D scene instance segmentation, 2022, arXiv:2211.15766.
[25]
VanDerHorn E., Mahadevan S., Digital twin: Generalization, characterization and implementation, Decis. Support Syst. 145 (2021),. URL: https://www.sciencedirect.com/science/article/pii/S0167923621000348.
[26]
Vu T., Kim K., Luu T.M., Nguyen X.T., Yoo C.D., SoftGroup for 3D instance segmentation on point clouds, 2022, https://doi.org/10.48550/ARXIV.2203.01509, URL: https://arxiv.org/abs/2203.01509.
[27]
Yin C., Wang B., Gan V.J., Wang M., Cheng J.C., Automated semantic segmentation of industrial point clouds using ResPointNet++, Autom. Constr. 130 (2021),. URL: https://www.sciencedirect.com/science/article/pii/S0926580521003253.
[28]
Zhang H., Wang T., An efficient method for producing deep learning point cloud datasets based on BIM 3D model and computer simulation, in: Xia J. (Ed.), Second International Symposium on Computer Technology and Information Science (ISCTIS 2022), vol. 12474, SPIE, International Society for Optics and Photonics, 2022,.
[29]
Zhong M., Chen X., Chen X., Zeng G., Wang Y., MaskGroup: Hierarchical point grouping and masking for 3D instance segmentation, in: 2022 IEEE International Conference on Multimedia and Expo (ICME), IEEE, 2022,.

Index Terms

  1. Semi-automated dataset creation for semantic and instance segmentation of industrial point clouds.
          Index terms have been assigned to the content through auto-classification.

          Recommendations

          Comments

          Please enable JavaScript to view thecomments powered by Disqus.

          Information & Contributors

          Information

          Published In

          cover image Computers in Industry
          Computers in Industry  Volume 155, Issue C
          Feb 2024
          251 pages

          Publisher

          Elsevier Science Publishers B. V.

          Netherlands

          Publication History

          Published: 14 March 2024

          Author Tags

          1. Scan-to-BIM
          2. Digital twin
          3. 3D semantic segmentation
          4. 3D instance segmentation
          5. Point cloud
          6. LiDAR

          Qualifiers

          • Research-article

          Contributors

          Other Metrics

          Bibliometrics & Citations

          Bibliometrics

          Article Metrics

          • 0
            Total Citations
          • 0
            Total Downloads
          • Downloads (Last 12 months)0
          • Downloads (Last 6 weeks)0
          Reflects downloads up to 09 Jan 2025

          Other Metrics

          Citations

          View Options

          View options

          Media

          Figures

          Other

          Tables

          Share

          Share

          Share this Publication link

          Share on social media