Editorial on Special Issue “3D Reconstruction and Mobile Mapping in Urban Environments Using Remote Sensing”
1
School of Computer Science, China University of Geosciences, Wuhan 430074, China
2
Department of Land Surveying and Geo-Informatics, The Hong Kong Polytechnic University, Hong Kong 999077, China
3
The College of Geodesy and Geomatics, Shandong University of Science and Technology, Qingdao 266590, China
4
State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, Wuhan 430074, China
*
Author to whom correspondence should be addressed.
Remote Sens. 2024, 16(17), 3236; https://doi.org/10.3390/rs16173236
Submission received: 16 August 2024
/
Accepted: 28 August 2024
/
Published: 31 August 2024
(This article belongs to the Special Issue 3D Reconstruction and Mobile Mapping in Urban Environments Using Remote Sensing)
1. Introduction
Both 3D reconstruction and mobile mapping are critical in supporting various applications in urban environments, including but not limited to autonomous driving, smart logistics, pedestrian navigation, and virtual reality. In the last decade, remote sensing-based techniques have emerged as a meaningful solution for ensuring urban environments are evaluated in an orderly fashion, due to the rapid evolution of cutting-edge techniques, e.g., SfM (Structure from Motion), SLAM (Simultaneous Localization and Mapping), and the revolution in deep learning techniques that enhance the entire pipeline, e.g., NeRF (Neural Radiance Field). In conclusion, the explosive development of 3D reconstruction and mobile mapping has been particularly notable in recent years.
This Special Issue comprises high-quality papers focusing on the techniques and applications of 3D reconstruction and mobile mapping in urban environments. A total of 15 papers are published in this Special Issue, covering topics such as image feature matching and dense matching, LiDAR/image-fused SLAM for image orientation and tunnel mapping, NeRF-based scene rendering and orthophoto generation, and other interesting applications, such as InSAR point cloud registration and 3D Ground-Penetrating Radar (3D GPR) for underground imaging and positioning. The details of each paper will be described in the following section.
2. Overview
Reliable feature matching is the first step of 3D reconstruction, determining the success of subsequent processing. Focusing on the feature matching of spherical images, Jiang et al. [1] present an algorithm by combining local geometric rectification with convolutional neural network (CNN) learned descriptors. It addresses the challenge of the geometric distortions inherent in spherical images and improves the performance of 3D reconstruction systems. The method utilized includes a local geometric rectification, a CNN-based descriptor learning network for rectified patches, and a robust essential matrix estimation for outlier removal. The effectiveness of the proposed solution is demonstrated through experiments using real spherical images.
Yao et al. [2] introduce a quasi-dense matching algorithm for oblique stereo images with large viewpoint changes. The core idea of the proposed method relies on the combination of VGG16-UNet-based semantic segmentation with LoFTR-based local feature enhancement. The method involves segmenting multiplanar scenes, performing affine-invariant feature matching, and enhancing weak texture regions to improve the matching accuracy. By using low-altitude stereo images, the experiments demonstrate significant advantages in match quantity, accuracy, and spatial distribution over classical and deep-learning methods.
The 3D reconstruction of ancient buildings plays a critical role in digital city construction. By using recent techniques, Ge et al. [3] present a NeRF-based 3D reconstruction workflow for UAV images with depth supervision. It introduces a multi-resolution hash coding approach to reduce hash conflicts and a truncated signed distance function (TSDF) to improve geometric accuracy. Through the use of collected UAV (Unmanned Aerial Vehicle) images, the test results demonstrate that the proposed solution can render images with clearer structural details and achieves a notable improvement in performance, with a 15% gain on average in the Peak Signal-to-Noise Ratio (PSNR) and a 10% gain in the Structural Similarity Index Measure (SSIM), producing detailed and accurate 3D models that are suitable for the digital preservation of cultural heritage sites.
A digital orthophoto is one of the most important developments, and it has been produced via the standard photogrammetric workflow for many years. Recently, by using the cutting-edge technique, Lv et al. [4] have presented a comparative study of explicit and implicit methods for generating digital orthophotos. The explicit method, termed as TDM (top-view-constrained dense matching), relies on the traditional geometric approach, while the implicit method, namely Instant NGP, employs neural rendering with Neural Radiance Fields. The comparative test concludes that both methods can produce accurate and high-quality orthophotos; due to the usage of the Compute Unified Device Architecture (CUDA) acceleration technique, TDM has significantly higher efficiency. To summarize, the study offers insights for selecting appropriate digital orthophoto generation methods based on efficiency and quality requirements.
Arza-García et al. [5] propose a cost-effective method for assessing the structural stability of a typical 3D model application, rubble mound breakwaters (RMBs), through the combination of UAV photogrammetry and Random Sample Consensus (RANSAC). In the proposed workflow, the photogrammetric point clouds of the RMB are generated via Structure from Motion and Multi-View Stereo (SfM-MVS) from pre- and post-storm flights, and they are fed to RANSAC for plane extraction and segmentation. Finally, by using a spatial proximity criterion, the cuboids of the two time periods are registered. The tests conducted on a breakwater in Porto, Portugal, show that the proposed method successfully identified post-storm structural changes and showcased its potential for monitoring RMB.
For urban 3D modeling, Cui et al. [6] introduce a method to extract urban building heights from Gaofen-7 stereo satellite images. The key technique involves using a contour matching algorithm to accurately determine rooftop elevations and using ground filtering to generate a DEM (Digital Elevation Model) from the DSM (Digital Surface Model). The proposed solution addresses challenges like occlusions, inaccurate ground elevation, and high-rise buildings, and it has been well-verified by using stereo images from three different provinces. The results verify the improved accuracy in building height extraction, especially beneficial for high-rise buildings and sites with complex terrain or vegetation.
For multi-source data fusion, Liu et al. [7] present a robust multi-sensor SLAM system, termed LVI-fusion, that integrates camera, lidar, and IMU data. The proposed mainly consists of a time alignment module to handle varying data frequencies, an image segmentation module for dynamic target removal, and a depth recovery model for feature points. The system uses a sliding window optimization module to achieve real-time pose calculation. The tests, carried out in various environments, demonstrate that the proposed method has high accuracy and robustness, and outperforms the other existing SLAM solutions, particularly in dynamic settings.
Xu et al. [8] present an enhanced Strapdown Inertial Navigation System (SINS) and a LiDAR tightly integrated SLAM system for urban environments with sparse structural features. The method refines an edge point extraction process from the LOAM algorithm and introduces a Kalman filter using line distance error as the primary observation metric to improve the robustness and accuracy of the system. The experimental tests conducted in various environments demonstrate its superior performance, with a 17% enhancement in positioning accuracy, especially in scenarios with limited structural features.
Point cloud registration, which aims to align two 3D point clouds using keypoint correspondences, is essential in photogrammetry and remote sensing. Traditional methods face challenges due to uncertainties in keypoint detection and matching, leading to outliers that reduce efficiency and accuracy. Wang et al. [9] present a new registration method using a compatibility graph and accelerated guided sampling, introducing a minimum subset sampling approach to minimize outlier impact and a preference-based sampling strategy to enhance computational efficiency and accuracy. Using synthetic and real datasets, the test results show that the proposed solution achieves a minimum rotation error of 0.737° and a minimum translation error of 0.0201 m, respectively, compared with existing methods.
In complex scenes with closely adjacent trees and buildings, the accurate extraction of building point clouds is challenging. Su et al. [10] introduce a two-stage method for building-point-cloud extraction based on geometric information. The first stage coarsely extracts building points, which are refined using mask polygons and a region-growing algorithm in the second stage. The method integrates the Alpha Shape algorithm and neighborhood expansion to address missing boundary points and applies mask extraction to the original points to avoid errors in facade identification. The approach shows significant improvements in extraction accuracy, outperforming PointNet by 20.73% in terms of precision and achieving results comparable to the HDL-JME-GGO network on the Urban-LiDAR and Vaihingen datasets.
Mutlti-source data fusion is a key step in the application of vehicle-borne mobile mapping systems (MMSs). Ji et al. [11] propose a method for vehicle-borne laser point cloud and panoramic images based on occlusion removal. The approach involves removing irrelevant points, extracting relevant scenes based on trajectory points, and applying a collinear model with spherical projection for matching. In addition, a vectorial angle selection algorithm is designed, in order to filter out occluded projections. The experimental results show the proposed solution can achieve an average pixel error of 2.82 pixels and a positional error of 4 cm, verifying that it is effective for data fusion applications in navigation, surveying, and mapping.
Cheng et al. [12] introduce an image-aided LiDAR framework for the extraction, classification, and characterization of lane markings from mobile mapping data. The framework addresses road safety by improving lane-marking inventory through a combination of imagery and LiDAR data, enhancing the detection of markings under various conditions. The framework includes road surface identification and color/intensity enhancement, and utilizes a geographic information system for visualization. The study demonstrates the system’s effectiveness over an extended road network, showing the potential to improve road safety analyses.
To address the need for the high-precision point cloud mapping of subway trains in long tunnel scenarios, Li et al. [13] introduce a LiDAR and inertial measurement sensor-based map construction method. The approach integrates a tightly coupled front-end odometry system by using Kalman filters with back-end optimization via factor graphs. In the front end, inertial measurements predict filter updates based on LiDAR points and local map planes. A global pose graph, built from inter-frame odometry and constraints, undergoes smoothing optimization for accurate mapping. The experiments show that it achieves a trajectory consistency of 0.1 m and an accumulated error of less than 0.2% compared to ground truth.
Array interferometric synthetic aperture radar (Array InSAR) systems can address shadow issues by performing scans in opposite directions. However, point clouds from two scans must be registered accurately. Cui et al. [14] present a robust registration method for urban Array InSAR point clouds, which uses images to represent 3D data, where pixel positions reflect azimuth and ground range, and pixel intensity represents height. The KAZE algorithm and an enhanced matching approach identify corresponding points to estimate transformation relationships. The experimental results show that it achieves the facade registration with a relative angular difference of less than 0.5°, and ground element registration achieves a Root Mean Square Error (RMSE) of less than 1.5 m.
Three-Dimensional Ground-Penetrating Radar (3D GPR) offers non-destructive and continuous subsurface detection but faces challenges regarding positioning accuracy in complex urban environments. Zhang et al. [15] propose a multi-level robust positioning method to enhance the accuracy of 3D GPR. In areas with strong GNSS signals, differential GNSS technology ensures rapid, precise positioning. For weak GNSS signals, a GNSS/INS tightly coupled solution improves accuracy, while in GNSS-denied environments, SLAM technology integrates INS data and 3D point clouds. This approach achieves a positioning accuracy of better than 10 cm, delivers high-quality 3D images of underground urban structures, and supports urban road surveys and underground disease detection.
3. Conclusions
This Special Issue focuses on the techniques for 3D reconstruction and mobile mapping in urban environments by using remote sensing methods, detailing the rapid development of new instruments for data acquisitions, the perspective invariant algorithms for feature matching, efficient SfM and SLAM-based solutions for image orientation, and deep-learning-based neural networks, all of which enhance the whole pipeline of 3D reconstruction and mobile mapping. Numerous high-quality manuscripts that covered a wide range of hot topics were submitted to this Special Issue. Ultimately, 15 of these papers were published after undergoing strict peer review, ensuring that this Special Issue will provide useful clues to guide further research.
Funding
This research received no external funding.
Acknowledgments
The authors would like to thank authors who have contributed their work to this Special Issue and reviewers who have paid their valuable time and effort to ensure the quality of all accepted papers. Meanwhile, heartfelt thanks to the Editors of this Special Issue, especially to Nicole Ju at MDPI for her careful and patient help in organizing this Special Issue.
Conflicts of Interest
The authors declare no conflicts of interest.
References
- Jiang, S.; Liu, J.; Li, Y.; Weng, D.; Chen, W. Reliable Feature Matching for Spherical Images via Local Geometric Rectification and Learned Descriptor. Remote Sens. 2023, 15, 4954. [Google Scholar] [CrossRef]
- Yao, G.; Zhang, J.; Zhu, F.; Gong, J.; Jin, F.; Fu, Q.; Ren, X. Quasi-Dense Matching for Oblique Stereo Images through Semantic Segmentation and Local Feature Enhancement. Remote Sens. 2024, 16, 632. [Google Scholar] [CrossRef]
- Ge, Y.; Guo, B.; Zha, P.; Jiang, S.; Jiang, Z.; Li, D. 3D Reconstruction of Ancient Buildings Using UAV Images and Neural Radiation Field with Depth Supervision. Remote Sens. 2024, 16, 473. [Google Scholar] [CrossRef]
- Lv, J.; Jiang, G.; Ding, W.; Zhao, Z. Fast Digital Orthophoto Generation: A Comparative Study of Explicit and Implicit Methods. Remote Sens. 2024, 16, 786. [Google Scholar] [CrossRef]
- Arza-García, M.; Gonçalves, J.A.; Ferreira Pinto, V.; Bastos, G. On-Site Stability Assessment of Rubble Mound Breakwaters Using Unmanned Aerial Vehicle-Based Photogrammetry and Random Sample Consensus. Remote Sens. 2024, 16, 331. [Google Scholar] [CrossRef]
- Cui, Y.; Zhao, S.; Jiang, W.; Yu, G. Urban Building Height Extraction from Gaofen-7 Stereo Satellite Images Enhanced by Contour Matching. Remote Sens. 2024, 16, 1556. [Google Scholar] [CrossRef]
- Liu, Z.; Li, Z.; Liu, A.; Shao, K.; Guo, Q.; Wang, C. LVI-Fusion: A Robust Lidar-Visual-Inertial SLAM Scheme. Remote Sens. 2024, 16, 1524. [Google Scholar] [CrossRef]
- Xu, X.; Guan, L.; Gao, Y.; Chen, Y.; Liu, Z. Enhanced Strapdown Inertial Navigation System (SINS)/LiDAR Tightly Integrated Simultaneous Localization and Mapping (SLAM) for Urban Structural Feature Weaken Occasions in Vehicular Platform. Remote Sens. 2024, 16, 2527. [Google Scholar] [CrossRef]
- Wang, C.; Zheng, Z.; Zha, B.; Li, H. Fast Robust Point Cloud Registration Based on Compatibility Graph and Accelerated Guided Sampling. Remote Sens. 2024, 16, 2789. [Google Scholar] [CrossRef]
- Su, Z.; Peng, J.; Feng, D.; Li, S.; Yuan, Y.; Zhou, G. A Building Point Cloud Extraction Algorithm in Complex Scenes. Remote Sens. 2024, 16, 1934. [Google Scholar] [CrossRef]
- Ji, J.; Wang, W.; Ning, Y.; Bo, H.; Ren, Y. Research on a Matching Method for Vehicle-Borne Laser Point Cloud and Panoramic Images Based on Occlusion Removal. Remote Sens. 2024, 16, 2531. [Google Scholar] [CrossRef]
- Cheng, Y.-T.; Shin, Y.-H.; Shin, S.-Y.; Koshan, Y.; Hodaei, M.; Bullock, D.; Habib, A. Image-Aided LiDAR Extraction, Classification, and Characterization of Lane Markings from Mobile Mapping Data. Remote Sens. 2024, 16, 1668. [Google Scholar] [CrossRef]
- Li, C.; Pan, W.; Yuan, X.; Huang, W.; Yuan, C.; Wang, Q.; Wang, F. High-Precision Map Construction in Degraded Long Tunnel Environments of Urban Subways. Remote Sens. 2024, 16, 809. [Google Scholar] [CrossRef]
- Cui, C.; Liu, Y.; Zhang, F.; Shi, M.; Chen, L.; Li, W.; Li, Z. A Novel Automatic Registration Method for Array InSAR Point Clouds in Urban Scenes. Remote Sens. 2024, 16, 601. [Google Scholar] [CrossRef]
- Zhang, J.; Hu, Q.; Zhou, Y.; Zhao, P.; Duan, X. A Multi-Level Robust Positioning Method for Three-Dimensional Ground Penetrating Radar (3D GPR) Road Underground Imaging in Dense Urban Areas. Remote Sens. 2024, 16, 1559. [Google Scholar] [CrossRef]
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
MDPI and ACS Style
Jiang, S.; Weng, D.; Liu, J.; Jiang, W. Editorial on Special Issue “3D Reconstruction and Mobile Mapping in Urban Environments Using Remote Sensing”. Remote Sens. 2024, 16, 3236. https://doi.org/10.3390/rs16173236
AMA Style
Jiang S, Weng D, Liu J, Jiang W. Editorial on Special Issue “3D Reconstruction and Mobile Mapping in Urban Environments Using Remote Sensing”. Remote Sensing. 2024; 16(17):3236. https://doi.org/10.3390/rs16173236
Chicago/Turabian StyleJiang, San, Duojie Weng, Jianchen Liu, and Wanshou Jiang. 2024. "Editorial on Special Issue “3D Reconstruction and Mobile Mapping in Urban Environments Using Remote Sensing”" Remote Sensing 16, no. 17: 3236. https://doi.org/10.3390/rs16173236
Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.