3D Point Cloud Generation Based on Multi-Sensor Fusion
<p>RTS smoothing algorithm flow.</p> "> Figure 2
<p>Flow chart of point cloud generation based on RTS filtering and smoothing.</p> "> Figure 3
<p>Track Mobile Laser Measurement System (TMLS).</p> "> Figure 4
<p>Schematic diagram of the coordinates of the main sensors of TMLS.</p> "> Figure 5
<p>Z + F 5016 scanner.</p> "> Figure 6
<p>FSINS3X.</p> "> Figure 7
<p>BW-VG527 Inclinometer.</p> "> Figure 8
<p>Program design structure diagram.</p> "> Figure 9
<p>Program interface.</p> "> Figure 10
<p>Scanner Calibration.</p> "> Figure 11
<p>Relationship between the plane coordinate systems of each sensor.</p> "> Figure 12
<p>Schematic diagram of inclination calibration.</p> "> Figure 13
<p>Trajectory correction.</p> "> Figure 14
<p>Displacement vector from the center of the target to the center of the track in the same section.</p> "> Figure 15
<p>Inclinometer roll angle and IMU roll angle.</p> "> Figure 16
<p>Recursive average filter effect.</p> "> Figure 17
<p>Point cloud generation overall effect.</p> "> Figure 18
<p>Test site (<b>left</b>) and target photos (<b>right</b>).</p> "> Figure 19
<p>Overall distribution of the target.</p> "> Figure 20
<p>Target photo (<b>left</b>) and Point cloud extraction target (<b>right</b>).</p> "> Figure 21
<p>Comparison of trajectory error before and after filter correction.</p> "> Figure 22
<p>Point cloud extraction target and target measurement position.</p> "> Figure 23
<p>Multi-sensor fusion generated point cloud and integrated navigation generated point cloud with the same name point selection.</p> ">
Abstract
:1. Introduction
2. Filtering and Smoothing Theory
2.1. Subsection Location Update Algorithm
2.2. Error Equation
2.3. RTS Smoothing Algorithm
3. A Method of 3D Point Cloud Generation by Multi-Sensor Fusion
3.1. Research on Track Mobile Laser Measurement System (TMLS)
3.1.1. Hardware Equipment
- (1)
- 3D Laser Scanner
- (2)
- Inertial measurement unit (IMU)
- (3)
- Inclination Sensor
3.1.2. System Software Design
3.1.3. System Calibration
- (1)
- Scanner Calibration
- (2)
- IMU Calibration
- (3)
- Horizontal inclination zero deviation and heading angle closure difference calibration
3.2. Trajectory Correction Algorithm
- (1)
- Track center point calculation
- (2)
- Original data estimation error
- (3)
- RTS filter correction
3.3. Attitude Angle Optimization Algorithm
- (1)
- Inclinometer correction
- (2)
- Recursive Averaging Filter
3.4. 3D Point Cloud Data Generation Method
4. Results
4.1. Experimental Overview
4.2. Filtered Smooth Trajectory Verification
4.3. Accuracy of Point Cloud Restoration
5. Discussion
6. Conclusions
Author Contributions
Funding
Data Availability Statement
Conflicts of Interest
References
- Puente, I.; González-Jorge, H.; Martínez-Sánchez, J.; Arias, P. Automatic detection of road tunnel luminaires using a mobile LiDAR system. Measurement 2014, 47, 569–575. [Google Scholar] [CrossRef]
- Zhou, Y.; Wang, S.; Mei, X.; Yin, W.; Lin, C.; Hu, Q.; Mao, Q. Railway Tunnel Clearance Inspection Method Based on 3D Point Cloud from Mobile Laser Scanning. Sensors 2017, 17, 2055. [Google Scholar] [CrossRef] [PubMed]
- Tran, T.H.; Taweep, C. Automated extraction of expressway road surface from mobile laser scanning data. J. Cent. South Univ. 2020, 27, 1917–1938. [Google Scholar] [CrossRef]
- Liu, S.; Sun, H.; Zhang, Z.; Li, Y.; Zhong, R.; Li, J.; Chen, S. A multi-scale deep feature for the instance segmentation of water leakages in tunnel using MLS point clouds. IEEE Trans. 2022, 60, 5702716. [Google Scholar]
- Sun, H.; Xu, Z.; Yao, L.; Zhong, R.; Du, L.; Wu, H. Tunnel Monitoring and Measuring System Using Mobile Laser Scanning: Design and Deployment. Remote Sens. 2020, 12, 730. [Google Scholar] [CrossRef]
- Chen, Z.; Li, Q.; Li, J.; Zhang, D.; Yu, J.; Yin, Y.; Lv, S.; Liang, A. IMU-Aided Registration of MLS Point Clouds Using Inertial Trajectory Error Model and Least Squares Optimization. Remote Sens. 2022, 14, 1365. [Google Scholar] [CrossRef]
- Jung, J.H.; Cha, J.; Chung, J.Y.; Kim, T.I.; Seo, M.H.; Park, S.Y.; Yeo, J.Y.; Park, C.G. Monocular Visual-Inertial-Wheel Odometry Using Low-Grade IMU in Urban Areas. IEEE Trans. 2022, 23, 925–938. [Google Scholar] [CrossRef]
- Du, L.; Zhong, R.; Sun, H.; Zhu, Q.; Zhang, Z. Study of the Integration of the CNU-TS-1 Mobile Tunnel Monitoring System. Sensors 2018, 18, 420. [Google Scholar] [CrossRef] [PubMed]
- Engstrand, A. Railway Surveying—A Case Study of the GRP 5000; Division of Geodesy and Geoinformatics Royal Institute of Technology (KTH): Stockholm, Sweden, 2011. [Google Scholar]
- Du, L.; Zhong, R.; Sun, H.; Liu, Y.; Wu, Q. Crossection positioning based on a dynamic MLS tunnel monitoring system. Photogramm. Rec. 2019, 34, 244–265. [Google Scholar] [CrossRef]
- Han, Y.; Sun, H.; Zhong, R. Three-Dimensional Linear Restoration of a Tunnel Based on Measured Track and Uncontrolled Mobile Laser Scanning. Sensors 2021, 21, 3815. [Google Scholar] [CrossRef] [PubMed]
- Sun, J.; Sun, H.; Zhong, R.; Han, Y. Deformation Detection Method of Mine Tunnel Based on Mobile Detection System. Sensors 2020, 20, 5400. [Google Scholar] [CrossRef] [PubMed]
- Li, L.; Zhang, X.; Lian, J.; Zhou, Y.; Zhen, W. Semantic SLAM algorithm based on road structural features. J. Harbin Inst. Technol. 2021, 53, 9. [Google Scholar]
- Karam, S.; Vosselman, G.; Peter, M.; Hosseinyalamdary, S.; Lehtola, V. Design, Calibration, and Evaluation of a Backpack Indoor Mobile Mapping System. Remote Sens. 2019, 11, 905. [Google Scholar] [CrossRef]
- Crassidis, J.L.; Junkins, J.L. Optimal Estimation of Dynamic Systems, 2nd ed.; Chapman & Hall/CRC Applied Mathematics & Nonlinear Science: Boca Raton, FL, USA, 2011. [Google Scholar]
- Fraser, D.; Potter, J. The optimum linear smoother as a combination of two optimum linear filters. IEEE Trans. 1969, 14, 387–390. [Google Scholar] [CrossRef]
- Fraser, D.; Potter, J. Massachusetts Institute of Technology, Cambridge, MA, USA. IEEE Trans. 1969, 14, 387–390. [Google Scholar]
- Ryall, T.G. Maximum Likelihood Estimates of Linear Dynamic System Parameters; Aeronautical Research Laboratories: Columbus, OH, USA, 1983. [Google Scholar]
- Teng, H.; Wang, Y.; Wang, L. Application Analysis of RTS Smoothing Filtering on the Data Post-Processing of BDS/INS Integrated Navigation. Int. Conf. Mechatron. Autom. Eng. 2017, 440–452. [Google Scholar] [CrossRef]
- Xuan, L.; Jing, Y. Application of RTS optimal smoothing algorithm in satellite attitude determination. In Proceedings of the 2011 2nd International Conference on Intelligent Control and Information Processing, Harbin, China, 25–28 July 2021; pp. 978–982. [Google Scholar]
- Jiang, Q. Research on High-Precision Inertial Measurement and Parameter Estimation Method Based on Smooth Filtering; National University of Defense Technology: Changsha, China, 2017. [Google Scholar]
- Titterton, D.; Weston, J. Strapdown Inertial Navigation Technology. Aerosp. Electron. Syst. Mag. IEEE 2004, 20, 33–34. [Google Scholar] [CrossRef]
- Seube, N.; Picard, A.; Rondeau, M. A simple method to recover the latency time of tactical grade IMU systems. ISPRS J. Photogramm. Remote Sens. 2012, 74, 85–89. [Google Scholar] [CrossRef]
- Chen, Q.; Zhang, Q.; Niu, X. Estimate the Pitch and Heading Mounting Angles of the IMU for Land Vehicular GNSS/INS Integrated System. IEEE Trans. 2020, 99, 1–13. [Google Scholar] [CrossRef]
- Li, Q.; Bai, Z.; Chen, B.; Guo, J.; Xin, H.; Cheng, Y.; Li, Q.; Wu, F. GNSS/INS multi-sensor integrated high-speed railway track measurement system. J. Surv. Mapp. 2020, 49, 569–579. [Google Scholar]
Parameter | Standard |
---|---|
Angle measurement accuracy | 14.4″ |
Distance measuring error | ≤1mm ± 10 ppm within range |
Vertical view | 320° |
Horizontal view | 360° |
Scan range | 0.3–365 m |
Maximum point rate | 1,100,000/s |
Rotating speed | Maximum 55 Hz |
Laser divergence | 0.3 mrad |
Reflection system | Fully enclosed rotating lens, built-in HDR camera and LED flash |
Index | Parameter | Remark | |
---|---|---|---|
Initial alignment | Horizontal attitude | 0.02° 0.05° (max) | horizontal attitude angle < 80° |
Course | 0.06° 0.10° | static start < 5 min dynamic start < 10 min | |
Navigation | attitude And Course hold | 0.01°/h | pure inertial navigation |
Positioning accuracy | 0.8 nm/h | pure inertial navigation | |
0.2%L (L is voyage) | combined odometer | ||
Maximum frequency of data transmission | 2000 Hz | ||
Other indicators | Operating temperature | −40–65 °C | |
Input voltage | 18~36VDC | ||
Input current | <0.6 A | ||
Power consumption | <14 W | ||
Electrical interface | RS422/CAN/NET | ||
Dimensions | 166 mm × 151 mm × 126 mm | ||
Weight | <4 kg |
Index | Parameter | |
---|---|---|
Attitude parameters | Dynamic accuracy | 0.1° |
Static accuracy | 0.01° | |
Resolution | 0.01° | |
Tilt range | ±90° | |
Other indicators | Maximum output frequency | 100 Hz |
Voltage | 9–35 V DC | |
Working current | 30 mA | |
Operating temperature | −40–85 °C | |
Electrical interface | RS232 | |
Weight | <360 g |
Mileage Location(m) | Deviation before Filtering Correction | Deviation after Filtering Correction | ||
---|---|---|---|---|
Lateral Deviation(m) | Vertical Deviation(m) | Lateral Deviation(m) | Vertical Deviation(m) | |
5 | 0.0347 | 0.0307 | 0.0347 | 0.0307 |
30 | −0.0145 | −0.0089 | 0.0052 | −0.0009 |
55 | −0.0633 | −0.0492 | 0.0086 | −0.0056 |
80 | −0.1116 | −0.0862 | 0.0060 | −0.0026 |
105 | −0.1523 | −0.1229 | 0.0028 | 0.0038 |
130 | −0.1644 | −0.1685 | 0.0080 | 0.0018 |
155 | −0.1794 | −0.2135 | 0.0021 | 0.0000 |
180 | −0.1916 | −0.2592 | 0.0312 | 0.0026 |
205 | −0.2538 | −0.3729 | 0.0482 | −0.0330 |
230 | −0.2150 | −0.3895 | 0.0440 | −0.0301 |
255 | −0.1420 | −0.3692 | −0.0420 | 0.0405 |
Average | 0.1384 | 0.1882 | 0.0239 | 0.0138 |
Target | North Error(m) | East Error(m) | Height Error(m) | Horizontal Deviation(m) | Elevation Deviation(m) |
---|---|---|---|---|---|
tag1 | 0.0107 | 0.0000 | 0.0000 | 0.0107 | 0.0000 |
tag2 | −0.0081 | 0.0008 | 0.0003 | 0.0081 | 0.0003 |
tag3 | −0.0107 | 0.0038 | 0.0005 | 0.0113 | 0.0005 |
tag4 | 0.0276 | 0.0129 | 0.0012 | 0.0305 | 0.0012 |
Average | 0.0143 | 0.0044 | 0.0005 | 0.0152 | 0.0005 |
Point ID | North Error(m) | East Error(m) | Height Error(m) | Horizontal Deviation(m) | Elevation Deviation(m) |
---|---|---|---|---|---|
1 | −0.0126 | 0.0105 | 0.0009 | 0.0164 | 0.0009 |
2 | 0.0081 | −0.0082 | 0.0031 | 0.0115 | 0.0031 |
3 | −0.0089 | 0.0294 | 0.0391 | 0.0307 | 0.0391 |
4 | 0.0072 | 0.0294 | −0.0054 | 0.0303 | 0.0054 |
5 | −0.0073 | 0.0341 | −0.0132 | 0.0348 | 0.0132 |
6 | −0.0089 | 0.0349 | −0.0199 | 0.0360 | 0.0199 |
Average | 0.0088 | 0.0244 | 0.0136 | 0.0266 | 0.0136 |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Han, Y.; Sun, H.; Lu, Y.; Zhong, R.; Ji, C.; Xie, S. 3D Point Cloud Generation Based on Multi-Sensor Fusion. Appl. Sci. 2022, 12, 9433. https://doi.org/10.3390/app12199433
Han Y, Sun H, Lu Y, Zhong R, Ji C, Xie S. 3D Point Cloud Generation Based on Multi-Sensor Fusion. Applied Sciences. 2022; 12(19):9433. https://doi.org/10.3390/app12199433
Chicago/Turabian StyleHan, Yulong, Haili Sun, Yue Lu, Ruofei Zhong, Changqi Ji, and Si Xie. 2022. "3D Point Cloud Generation Based on Multi-Sensor Fusion" Applied Sciences 12, no. 19: 9433. https://doi.org/10.3390/app12199433