Abstract
Scan-to-BIM systems convert image and point cloud data into accurate 3D models of buildings. Research on Scan-to-BIM has largely focused on the automated identification of structural components. However, design and maintenance projects require information on a range of other assets including mechanical, electrical, and plumbing (MEP) components. This paper presents a deep learning solution that locates and labels MEP components in 360\(^{\circ }\) images and phone images, specifically sockets, switches and radiators. The classification and location data generated by this solution could add useful context to BIM models. The system developed for this project uses transfer learning to retrain a Faster Region-based Convolutional Neural Network (Faster R-CNN) for the MEP use case. The performance of the neural network across image formats is investigated. A dataset of 249 360\(^{\circ }\) images and 326 phone images was built to train the deep learning model. The Faster R-CNN achieved high precision and comparatively low recall across all image formats.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Adán, A., Quintana, B., Prieto, S.A., Bosché, F.: Scan-to-BIM for ‘secondary’ building components. Adv. Eng. Inf. 37, 119–138 (2018)
Ahmed, M.F., Haas, C.T., Haas, R.: Automatic detection of cylindrical objects in built facilities. J. Comput. Civ. Eng. 28(3), 04014009 (2014)
Babacan, K., Chen, L., Sohn, G.: Semantic segmentation of indoor point clouds using convolutional neural network. ISPRS Annals Photogramm. Remote Sens. Spat. Inf. Sci. 4, 101–108 (2017)
Barazzetti, L., Previtali, M., Roncoroni, F.: Can we use low-cost 360 degree cameras to create accurate 3D models? Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci. 42(2), 69–75 (2018)
Bassier, M., Vergauwen, M., Van Genechten, B.: Automated classification of heritage buildings for as-built BIM using machine learning techniques. In: ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, vol. 4, pp. 25–30. Copernicus GmbH (2017)
Bosché, F., Ahmed, M., Turkan, Y., Haas, C.T., Haas, R.: The value of integrating Scan-to-BIM and Scan-vs-BIM techniques for construction monitoring using laser scanning and BIM: the case of cylindrical MEP components. Autom. Constr. 49, 201–213 (2015)
Chen, J., Kira, Z., Cho, Y.K.: Deep learning approach to point cloud scene understanding for automated scan to 3D reconstruction. J. Comput. Civ. Eng. 33(4), 04019027 (2019)
COCO Dataset: Detection Evaluation (2019). http://cocodataset.org/#detection-eval
Czerniawski, T., Nahangi, M., Haas, C., Walbridge, S.: Pipe spool recognition in cluttered point clouds using a curvature-based shape descriptor. Autom. Constr. 71(2), 346–358 (2016)
Díaz-Vilariño, L., González-Jorge, H., Martínez-Sánchez, J., Lorenzo, H.: Automatic LiDAR-based lighting inventory in buildings. Meas. J. Int. Meas. Conf. 73, 544–550 (2015). https://doi.org/10.1016/j.measurement.2015.06.009
Dimitrov, A., Golparvar-Fard, M.: Segmentation of building point cloud models including detailed architectural/structural features and MEP systems. Autom. Constr. 51, 32–45 (2015)
Eruhimov, V., Meeussen, W.: Outlet detection and pose estimation for robot continuous operation. In: 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 2941–2946. Institute of Electrical and Electronics Engineers (IEEE), December 2011
Fan, Q., Brown, L., Smith, J.: A closer look at Faster R-CNN for vehicle detection. In: IEEE Intelligent Vehicles Symposium, Proceedings, pp. 124–129. Institute of Electrical and Electronics Engineers Inc., August 2016
Gao, T., Packer, B., Koller, D.: A segmentation-aware object detection model with occlusion handling. In: CVPR 2011, pp. 1361–1368. IEEE (2011)
Hamledari, H., McCabe, B., Davari, S.: Automated computer vision-based detection of components of under-construction indoor partitions. Autom. Constr. 74, 78–94 (2017)
Hosang, J., Benenson, R., Dollár, P., Schiele, B.: What makes for effective detection proposals?. Technical report, Max Planck Institute for Informatics (2015)
Huang, J., You, S.: Detecting objects in scene point cloud: a combinational approach. In: 2013 International Conference on 3D Vision, pp. 175–182 (2013)
Huang, J., et al.: Speed/accuracy trade-offs for modern convolutional object detectors. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7310–7311. The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach (2017)
Kim, P., Chen, J., Cho, Y.K.: Robotic sensing and object recognition from thermal-mapped point clouds. Int. J. Intell. Robot. Appl. 1(3), 243–254 (2017). https://doi.org/10.1007/s41315-017-0023-9
Krispel, U., Evers, H.L., Tamke, M., Viehauser, R., Fellner, D.W.: Automatic Texture and Orthophoto Generation From Registered Panoramic Views. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences (2015)
Kropp, C., Koch, C., König, M.: Interior construction state recognition with 4D BIM registered image sequences. Autom. Constr. 86, 11–32 (2018)
Lee, M., Kim, H., Paik, J.: Correction of barrel distortion in fisheye lens images using image-based estimation of distortion parameters. IEEE Access 7, 45723–45733 (2019)
Li, J., Liang, X., Wei, Y., Xu, T., Feng, J., Yan, S.: Perceptual generative adversarial networks for small object detection. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1951–1959 (2017)
Liang, Z., Shao, J., Zhang, D., Gao, L.: Small object detection using deep feature pyramid networks. In: Hong, R., Cheng, W.-H., Yamasaki, T., Wang, M., Ngo, C.-W. (eds.) PCM 2018. LNCS, vol. 11166, pp. 554–564. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00764-5_51
Lin, T.-Y., et al.: Microsoft COCO: common objects in context. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8693, pp. 740–755. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10602-1_48
Lopes, R., Yin, D., Poole, B., Gilmer, J., Cubuk, E.D.: Improving robustness without sacrificing accuracy with patch Gaussian augmentation. ArXiv (2019)
Maalek, R., Lichti, D.D., Ruwanpura, J.Y.: Automatic recognition of common structural elements from point clouds for automated progress monitoring and dimensional quality control in reinforced concrete construction. Remote Sens. 11(9), 1102 (2019)
Maji, S., Malik, J.: Object detection using a max-margin Hough transform. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1038–1045. Institute of Electrical and Electronics Engineers (IEEE), March 2009
Meeussen, W., et al.: Autonomous door opening and plugging in with a personal robot. In: Proceedings - IEEE International Conference on Robotics and Automation, pp. 729–736 (2010)
Michie, D.: “memo” functions and machine learning. Nature 218(5136), 19–22 (1968)
Perez, L., Wang, J.: The effectiveness of data augmentation in image classification using deep learning. ArXiv, December 2017
Quintana, B., Prieto, S.A., Adán, A., Bosché, F.: Door detection in 3D coloured point clouds of indoor environments. Autom. Constr. 85, 146–166 (2018)
Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans. Pattern Anal. Mach. Intell. 39, 1137–1149 (2015). https://doi.org/10.1109/TPAMI.2016.2577031
Ren, Y., Zhu, C., Xiao, S.: Small object detection in optical remote sensing images via modified faster R-CNN. Appl. Sci. 8(5), 813 (2018)
Shorten, C., Khoshgoftaar, T.M.: A survey on image data augmentation for deep learning. J. Big Data 6(1), 60 (2019)
Son, H., Kim, C., Kim, C.: Fully automated as-built 3d pipeline extraction method from laser-scanned data based on curvature computation. J. Comput. Civ. Eng. 29(4), B4014003 (2015)
Song, L., Gong, D., Li, Z., Liu, C., Liu, W.: Occlusion robust face recognition based on mask learning with pairwise differential Siamese network. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 773–782 (2019)
Turcsany, D., Mouton, A., Breckon, T.P.: Improving feature-based object recognition for X-ray baggage security screening using primed visual words. In: Proceedings of the IEEE International Conference on Industrial Technology, pp. 1140–1145 (2013)
Wu, R., Yan, S., Shan, Y., Dang, Q., Sun, G.: Deep image: scaling up image recognition. arXiv preprint arXiv:1501.02876 7(8) (2015)
Acknowledgements
This work was conducted as part of the BIMERR project has received funding from the European Commission’s Horizon 2020 research and innovation programme under grant agreement No 820621. The views and opinions expressed in this article are those of the authors and do not necessarily reflect any official policy or position of the European Commission.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Kufuor, J., Mohanty, D.D., Valero, E., Bosché, F. (2021). Automatic MEP Component Detection with Deep Learning. In: Del Bimbo, A., et al. Pattern Recognition. ICPR International Workshops and Challenges. ICPR 2021. Lecture Notes in Computer Science(), vol 12667. Springer, Cham. https://doi.org/10.1007/978-3-030-68787-8_28
Download citation
DOI: https://doi.org/10.1007/978-3-030-68787-8_28
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-68786-1
Online ISBN: 978-3-030-68787-8
eBook Packages: Computer ScienceComputer Science (R0)