Abstract
Nowadays there is an increasing need for using artificial intelligence techniques in image-based documentation and survey in archeology, architecture or civil engineering applications. Brick segmentation is an important initial step in the documentation and analysis of masonry wall images. However, due to the heterogeneous material, size, shape and arrangement of the bricks, it is highly challenging to develop a widely adoptable solution for the problem via conventional geometric and radiometry based approaches. In this paper, we propose a new technique which combines the strength of deep learning for brick seed localization, and the Watershed algorithm for accurate instance segmentation. More specifically, we adopt a U-Net-based delineation algorithm for robust marker generation in the Watershed process, which provides as output the accurate contours of the individual bricks, and also separates them from the mortar regions. For training the network and evaluating our results, we created a new test dataset which consist of 162 hand-labeled images of various wall categories. Quantitative evaluation is provided both at instance and at pixel level, and the results are compared to two reference methods proposed for wall delineation, and to a morphology based brick segmentation approach. The experimental results showed the advantages of the proposed U-Net markered Watershed method, providing average F1-scores above 80%.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Bai, M., Urtasun, R.: Deep watershed transform for instance segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, Hawaii, pp. 5221–5229 (2017)
Bosché, F., Valero, E., Forster, A., Wilson, L., Leslie, A.: Evaluation of historic masonry substrates: towards greater objectivity and efficiency, June 2016. https://doi.org/10.4324/9781315628011-8
Llamas, J., Lerones, P.M., Zalama, E., Gómez-García-Bermejo, J.: Applying deep learning techniques to cultural heritage images within the INCEPTION project. In: Ioannides, M., et al. (eds.) EuroMed 2016. LNCS, vol. 10059, pp. 25–32. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-48974-2_4
Hemmleb, M., Weritz, A.F., Schiemenz, B.A., Grote, C.A., Maierhofer, C.: Multi-spectral data acquisition and processing techniques for damage detection on building surfaces. In: ISPRS Commission V Symposium, pp. 1–6, January 2006
Kuhn, H.W.: The Hungarian method for the assignment problem. Naval Res. Logistic Q. 2, 83–97 (1955)
Llamas, J.M., Lerones, P., Medina, R., Zalama, E., Gómez-García-Bermejo, J.: Classification of architectural heritage images using deep learning techniques. Appl. Sci. 7, 992 (2017). https://doi.org/10.3390/app7100992
Muñoz, X., Freixenet, J., Cufi, X., Marti, J.: Strategies for image segmentation combining region and boundary information. Pattern Recogn. Lett. 24, 375–392 (2003). https://doi.org/10.1016/S0167-8655(02)00262-3
Oses, N., Dornaika, F., Moujahid, A.: Image-based delineation and classification of built heritage masonry. Remote Sensing 6(3), 1863–1889 (2014). https://doi.org/10.3390/rs6031863
Riveiro, B., Conde, B., Gonzalez, H., Arias, P., Caamaño, J.: Automatic creation of structural models from point cloud data: the case of masonry structures. ISPRS Ann. Photogrammetry Remote Sens. Spat. Inf. Sci. II–3/W5, 3–9 (2015). https://doi.org/10.5194/isprsannals-II-3-W5-3-2015
Roerdink, J.B., Meijster, A.: The watershed transform: definitions, algorithms and parallelization strategies. Fundam. Inf. 41(1–2), 187–228 (2000). http://dl.acm.org/citation.cfm?id=2372488.2372495
Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Sithole, G.: Detection of bricks in a masonry wall. Int. Arch. Photogrammetry Remote Sens. Spat. Inf. Sci. XXXVII, 567–572 (2008)
Tao, Y., Palasek, P., Ling, Z., Patras, I.: Background modelling based on generative Unet. In: IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), pp. 1–6, August 2017. https://doi.org/10.1109/AVSS.2017.8078483
Zhang, K., et al.: Multi-scale colorectal tumour segmentation using a novel coarse to fine strategy. In: Proceedings of the British Machine Vision Conference (BMVC), pp. 97.1–97.12, September 2016. https://doi.org/10.5244/C.30.97
Zyuzin, V., et al.: Identification of the left ventricle endocardial border on two-dimensional ultrasound images using the convolutional neural network Unet. In: 2018 Ural Symposium on Biomedical Engineering, Radioelectronics and Information Technology (USBEREIT), pp. 76–78, May 2018. https://doi.org/10.1109/USBEREIT.2018.8384554
Acknowledgement
This work was supported by the National Research, Development and Innovation Fund (grants NKFIA K-120233 and KH-125681), and by the Szechenyi 2020 Program (grants EFOP-3.6.2-16-2017-00013 and 3.6.3-VEKOP-16- 2017-00002). Budapest, Hungary.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Ibrahim, Y., Nagy, B., Benedek, C. (2019). CNN-Based Watershed Marker Extraction for Brick Segmentation in Masonry Walls. In: Karray, F., Campilho, A., Yu, A. (eds) Image Analysis and Recognition. ICIAR 2019. Lecture Notes in Computer Science(), vol 11662. Springer, Cham. https://doi.org/10.1007/978-3-030-27202-9_30
Download citation
DOI: https://doi.org/10.1007/978-3-030-27202-9_30
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-27201-2
Online ISBN: 978-3-030-27202-9
eBook Packages: Computer ScienceComputer Science (R0)