Abstract
Automatic mapping of individual urban trees is increasingly important to city administration and planing. Although deep learning algorithms are now standard methodology in computer vision, their adaption to individual tree detection in urban areas has hardly been investigated so far. In this work, we propose a deep single-shot object detection network to find urban trees in point clouds from airborne laser scanning. The network consists of a sparse 3D convolutional backbone for feature extraction and a subsequent single-shot region proposal network for the actual detection. It takes as input raw 3D voxel clouds, discretized from the point cloud in preprocessing. Outputs are cylindrical tree objects paired with their detection scores. We train and evaluate the network on the ISPRS Vaihingen 3D Benchmark dataset with custom tree object labels. The general feasibility of our approach is demonstrated. It achieves promising results compared to a traditional 2D baseline using watershed segmentation. We also conduct comparisons with state-of-the-art machine learning methods for semantic point segmentation.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Bulatov, D., Wayand, I., Schilling, H.: Automatic tree-crown detection in challenging scenarios. In: ISPRS Archives, vol. XLI-B3, pp. 575–582 (2016)
Chen, X., Ma, H., Wan, J., Li, B., Xia, T.: Multi-view 3D object detection network for autonomous driving. In: CVPR, pp. 6526–6534 (2017)
City of Melville Council: Urban forest strategic plan 2017–2036: Plan a: City-controlled plan. Technical report, City of Melville, Perth, Australia (2017)
Eysn, L., et al.: A benchmark of lidar-based single tree detection methods using heterogeneous forest data from the alpine space. Forests 6(5), 1721–1747 (2015)
Geiger, A., Lenz, P., Urtasun, R.: Are we ready for autonomous driving? The Kitti vision benchmark suite. In: CVPR, pp. 3354–3361 (2012)
Girshick, R.: Fast R-CNN. In: ICCV, pp. 1440–1448 (2015)
Graham, B., Engelcke, M., van der Maaten, L.: 3D semantic segmentation with submanifold sparse convolutional networks. In: CVPR, pp. 9224–9232 (2018)
Haala, N., Kölle, M., Cramer, M., Laupheimer, D., Mandlburger, G., Glira, P.: Hybrid georeferencing, enhancement and classification of ultra-high resolution UAV lidar and image point clouds for monitoring applications. ISPRS Ann. V-2-2020, 727–734 (2020)
Hirschmugl, M., Ofner, M., Raggam, J., Schardt, M.: Single tree detection in very high resolution remote sensing data. Remote Sens. Environ. 110(4), 533–544 (2007). ForestSAT Special Issue
Hu, P., Held, D., Ramanan, D.: Learning to optimally segment point clouds. IEEE Robot. Autom. Lett. 5(2), 875–882 (2020)
Höfle, B., Hollaus, M., Hagenauer, J.: Urban vegetation detection using radiometrically calibrated small-footprint full-waveform airborne LiDAR data. ISPRS J. Photogram. Remote Sens. 67, 134–147 (2012)
Kaartinen, H., et al.: An international comparison of individual tree detection and extraction using airborne laser scanning. Remote Sens. 4(4), 950–974 (2012)
Koch, B., Heyder, U., Weinacker, H.: Detection of individual tree crowns in airborne lidar data. Photogram. Eng. Remote Sens. 72(4), 357–363 (2006)
Kölle, M., Walter, V., Schmohl, S., Soergel, U.: Hybrid acquisition of high quality training data for semantic segmentation of 3D point clouds using crowd-based active learning. ISPRS Ann. V-2-2020, 501–508 (2020)
Ku, J., Mozifian, M., Lee, J., Harakeh, A., Waslander, S.L.: Joint 3D proposal generation and object detection from view aggregation. In: IROS, pp. 1–8 (2018)
Lang, A.H., Vora, S., Caesar, H., Zhou, L., Yang, J., Beijbom, O.: Pointpillars: fast encoders for object detection from point clouds. In: CVPR, pp. 12689–12697 (2019)
Li, B.: 3D fully convolutional network for vehicle detection in point cloud. In: IROS, pp. 1513–1518 (2017)
Li, X., Chen, W.Y., Sanesi, G., Lafortezza, R.: Remote sensing in urban forestry: recent applications and future directions. Remote Sens. 11(10) (2019)
Liew, S.C., Huang, X., Lin, E.S., Shi, C., Yee, A.T.K., Tandon, A.: Integration of tree database derived from satellite imagery and LiDAR point cloud data. ISPRS Arch. XLII-4/W10, 105–111 (2018)
Liu, W., et al.: SSD: single shot multibox detector. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9905, pp. 21–37. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46448-0_2
Man, Q., Dong, P., Yang, X., Wu, Q., Han, R.: Automatic extraction of grasses and individual trees in urban areas based on airborne hyperspectral and lidar data. Remote Sens. 12(17) (2020)
Niemeyer, J., Rottensteiner, F., Soergel, U.: Contextual classification of lidar data and building object detection in urban areas. ISPRS J. 87, 152–165 (2014)
Persson, A.: Extraction of individual trees using laser radar data. Technical report, Swedish Defence Research Agency (2001)
Pfeifer, N., Mandlburger, G., Otepka, J., Karel, W.: OPALS - a framework for airborne laser scanning data analysis. Comput. Environ. Urban Syst. 45, 125–136 (2014)
Pleoianu, A.I., Stupariu, M.S., andric, I., Pătru-Stupariu, I., Drăguț L.: Individual tree-crown detection and species classification in very high-resolution remote sensing imagery using a deep learning ensemble model. Remote Sens. 12(15) (2020)
Qi, C.R., Litany, O., He, K., Guibas, L.: Deep Hough voting for 3D object detection in point clouds. In: ICCV, pp. 9276–9285 (2019)
Qi, C.R., Yi, L., Su, H., Guibas, L.J.: PointNet++: deep hierarchical feature learning on point sets in a metric space. In: NIPS, pp. 5105–5114 (2017)
Redmon, J., Farhadi, A.: YOLOv3: an incremental improvement. CoRR abs/1804.02767 (2018)
Reitberger, J.: 3D Segmentierung von Einzelbäumen und Baumartenklasifikation aus Daten flugzeuggetragener Full Waveform Laserscanner. Ph.D. thesis, Fakultät für Bauingenieur- und Vermessungswesen der Technischen Universität München (2010)
Ren, M., Pokrovsky, A., Yang, B., Urtasun, R.: SBNet: sparse blocks network for fast inference. In: CVPR, pp. 8711–8720 (2018)
Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. IEEE PAMI 39(6), 1137–1149 (2017)
Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Schmohl, S., Soergel, U.: Submanifold sparse convolutional networks for semantic segmentation of large-scale ALS point clouds. In: ISPRS Annals, vol. IV-2/W5, pp. 77–84 (2019)
Shi, S., Wang, X., Li, H.: PointRCNN: 3D object proposal generation and detection from point cloud. In: CVPR, pp. 770–779 (2019)
Shi, S., Wang, Z., Wang, X., Li, H.: From points to parts: 3D object detection from point cloud with part-aware and part-aggregation network. CoRR abs/1907.03670 (2019)
Simon, M., Milz, S., Amende, K., Gross, H.-M.: Complex-YOLO: an Euler-region-proposal for real-time 3D object detection on point clouds. In: Leal-Taixé, L., Roth, S. (eds.) ECCV 2018. LNCS, vol. 11129, pp. 197–209. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-11009-3_11
Song, S., Lichtenberg, S.P., Xiao, J.: Sun RGB-D: a RGB-D scene understanding benchmark suite. In: CVPR, pp. 567–576 (2015)
Wolf (né Straub), B.M., Heipke, C.: Automatic extraction and delineation of single trees from remote sensing data. Mach. Vis. Appl. 18(5), 317–330 (2007)
Walter, V., Kölle, M., Yin, Y.: Evaluation and optimisation of crowd-based collection of trees from 3D point clouds. ISPRS Ann. V-4-2020, 49–56 (2020)
Wang, D.Z., Posner, I.: Voting for voting in online point cloud object detection. In: Proceedings of Robotics: Science and Systems (2015)
Wegner, J.D., Branson, S., Hall, D., Schindler, K., Perona, P.: Cataloging public objects using aerial and street-level images - urban trees. In: CVPR, pp. 6014–6023 (2016)
Weinmann, M., Weinmann, M., Mallet, C., Brédif, M.: A classification-segmentation framework for the detection of individual trees in dense mms point cloud data acquired in urban areas. Remote Sens. 9(3) (2017)
Winiwarter, L., Mandlburger, G., Schmohl, S., Pfeifer, N.: Classification of ALS point clouds using end-to-end deep learning. PFG 87(3), 75–90 (2019)
Xie, Y., Bao, H., Shekhar, S., Knight, J.: A timber framework for mining urban tree inventories using remote sensing datasets. In: ICDM, pp. 1344–1349 (2018)
Yan, Y., Mao, Y., Li, B.: Second: sparsely embedded convolutional detection. Sensors 18(10) (2018)
Yang, Z., Sun, Y., Liu, S., Shen, X., Jia, J.: STD: sparse-to-dense 3D object detector for point cloud. In: ICCV, pp. 1951–1960 (2019)
Zhang, C., Zhou, Y., Qiu, F.: Individual tree segmentation from lidar point clouds for urban forest inventory. Remote Sens. 7(6), 7892–7913 (2015)
Zhao, R., Pang, M., Wang, J.: Classifying airborne lidar point clouds via deep features learned by a multi-scale convolutional neural network. Int. J. Geogr. Inf. Sci. 32(5), 960–979 (2018)
Zhou, Y., Tuzel, O.: VoxelNet: end-to-end learning for point cloud based 3D object detection. In: CVPR, pp. 4490–4499 (2018)
Zhu, B., Jiang, Z., Zhou, X., Li, Z., Yu, G.: Class-balanced grouping and sampling for point cloud 3D object detection. CoRR abs/1908.09492 (2019)
Acknowledgements
The Vaihingen dataset was provided by the German Society for Photogrammetry, Remote Sensing and Geoinformation (DGPF).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Schmohl, S., Kölle, M., Frolow, R., Soergel, U. (2021). Towards Urban Tree Recognition in Airborne Point Clouds with Deep 3D Single-Shot Detectors. In: Del Bimbo, A., et al. Pattern Recognition. ICPR International Workshops and Challenges. ICPR 2021. Lecture Notes in Computer Science(), vol 12667. Springer, Cham. https://doi.org/10.1007/978-3-030-68787-8_38
Download citation
DOI: https://doi.org/10.1007/978-3-030-68787-8_38
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-68786-1
Online ISBN: 978-3-030-68787-8
eBook Packages: Computer ScienceComputer Science (R0)