Information Sensing Technology for Intelligent/Driverless Vehicle

Topic Information

Dear Colleagues,

As the basis for vehicle positioning and path planning, the environmental perception system is a significant part of intelligent/driverless vehicles, which is used to get the environmental information around the vehicle including roads, obstacles, traffic signs, and the vital signs of the driver. In the past few years, environmental perception technology based on various vehicle-mounted sensors (camera, laser, millimeter-wave radar, and GPS/IMU) has made rapid progress. With the further research of automatic driving and assisted driving, the information sensing technology of driverless cars has become a research hotspot, and thus the performance of the vehicle-mounted sensors should be improved to adapt to the complex driving environment in our daily life. However, in reality, there are still many development issues, such as the technology not being mature, the instrument not being advanced, and the experiment environment not being real. All these problems pose great challenges to the traditional vehicle-mounted sensor system and information perception technology. In general, it motivates the need for new environmental perception systems, signal processing methods, and even new types of sensors.

This topic is devoted to highlighting the most advanced studies in technology, methodology, and applications of sensors mounted on intelligent/unmanned driving vehicle. Papers dealing with fundamental theoretical analyses as well as those demonstrating their application to real-world and/or emerging problems will be welcome. The journal publishes original papers, and from time to time invited review articles, in all areas related to the sensors mounted on intelligent/unmanned driving vehicles including, but not limited to, the following suggested topics:

• Vehicle-mounted millimeter-wave radar technology;
• Vehicle-mounted LiDAR technology;
• Vehicle visual sensors;
• High-precision positioning technology based on GPS/IMU;
• Muti-sensor data fusion (MSDF);
• New sensor systems mounted on intelligent/unmanned vehicle.

Dr. Shiyang Tang
Dr. Zhanye Chen
Dr. Yan Huang
Dr. Ping Guo
Topic Editors

Keywords

Participating Journals

Journal Name	Impact Factor	CiteScore	Launched Year	First Decision (median)	APC
Remote Sensing remotesensing	4.2	8.3	2009	24.7 Days	CHF 2700
Sensors sensors	3.4	7.3	2001	16.8 Days	CHF 2600
Geomatics geomatics	-	-	2021	21.8 Days	CHF 1000
Smart Cities smartcities	7.0	11.2	2018	25.8 Days	CHF 2000
Vehicles vehicles	2.4	4.1	2019	24.7 Days	CHF 1600

MDPI Topics is cooperating with Preprints.org and has built a direct connection between MDPI journals and Preprints.org. Authors are encouraged to enjoy the benefits by posting a preprint at Preprints.org prior to publication:

1. Immediately share your ideas ahead of publication and establish your research priority;
2. Protect your idea from being stolen with this time-stamped preprint article;
3. Enhance the exposure and impact of your research;
4. Receive feedback from your peers in advance;
5. Have it indexed in Web of Science (Preprint Citation Index), Google Scholar, Crossref, SHARE, PrePubMed, Scilit and Europe PMC.

Related Topic

Published Papers (29 papers)

Order results

Content type Publication Date

Result details

Normal Extended Compact

Journals

All Remote Sensing Sensors Geomatics Smart Cities Vehicles

Show export options expand_more Show export options expand_less

Select all

Export citation of selected articles as:

Plain Text BibTeX BibTeX (without abstracts) Endnote Endnote (without abstracts) Tab-delimited RIS

Error

Oops... you haven't selected anything for export.

Ok

clear

15 pages, 4087 KiB  

Open AccessArticle

Improving the Efficiency of 3D Monocular Object Detection and Tracking for Road and Railway Smart Mobility

by Alexandre Evain, Antoine Mauri, François Garnier, Messmer Kounouho, Redouane Khemmar, Madjid Haddad, Rémi Boutteau, Sébastien Breteche and Sofiane Ahmedali

Sensors 2023, 23(6), 3197; https://doi.org/10.3390/s23063197 - 16 Mar 2023

Cited by 3 | Viewed by 2048

Abstract

Three-dimensional (3D) real-time object detection and tracking is an important task in the case of autonomous vehicles and road and railway smart mobility, in order to allow them to analyze their environment for navigation and obstacle avoidance purposes. In this paper, we improve [...] Read more.

Three-dimensional (3D) real-time object detection and tracking is an important task in the case of autonomous vehicles and road and railway smart mobility, in order to allow them to analyze their environment for navigation and obstacle avoidance purposes. In this paper, we improve the efficiency of 3D monocular object detection by using dataset combination and knowledge distillation, and by creating a lightweight model. Firstly, we combine real and synthetic datasets to increase the diversity and richness of the training data. Then, we use knowledge distillation to transfer the knowledge from a large, pre-trained model to a smaller, lightweight model. Finally, we create a lightweight model by selecting the combinations of width, depth & resolution in order to reach a target complexity and computation time. Our experiments showed that using each method improves either the accuracy or the efficiency of our model with no significant drawbacks. Using all these approaches is especially useful for resource-constrained environments, such as self-driving cars and railway systems. Full article

(This article belongs to the Topic Information Sensing Technology for Intelligent/Driverless Vehicle)

►▼ Show Figures

Figure 1

Figure 1
<p>Object tracking with 3D bounding boxes and tracking numbers.</p> Full article ">Figure 2
<p>Visualization of 2D object tracking and predicted object trajectories. The numbers correspond to the tracking number of the detected vehicles, while their color corresponds to the position of the vehicles.</p> Full article ">Figure 3
<p>The Citroën AMI modified by the ESIGELEC for data recordings.</p> Full article ">Figure 4
<p>RTMaps diagram. We used the library “Python Bridge” of RTMaps in order to integrate the Yolov5 algorithm on the Jetson TX2.</p> Full article ">Figure 5
<p>Inference results comparison using the previous models.</p> Full article ">Figure 6
<p>Comparison of the training on KITTI between a distilled and a classic model.</p> Full article ">Figure 7
<p>The algorithm tested in real-time conditions on an embedded Jetson.</p> Full article ">

39 pages, 19855 KiB  

Open AccessArticle

A Localization Algorithm Based on Global Descriptor and Dynamic Range Search

by Yongzhe Chen, Gang Wang, Wei Zhou, Tongzhou Zhang and Hao Zhang

Remote Sens. 2023, 15(5), 1190; https://doi.org/10.3390/rs15051190 - 21 Feb 2023

Cited by 2 | Viewed by 2127

Abstract

The map-based localization method is considered an effective supplement to the localization under the GNSS-denied environment. However, since the map is constituted by the dispersed keyframes, it sometimes happens that the initial position of the unmanned ground vehicle (UGV) lies between the map [...] Read more.

The map-based localization method is considered an effective supplement to the localization under the GNSS-denied environment. However, since the map is constituted by the dispersed keyframes, it sometimes happens that the initial position of the unmanned ground vehicle (UGV) lies between the map keyframes or is not on the mapping trajectory. In both cases, it will be impossible to precisely estimate the pose of the vehicle directly via the relationship between the current frame and the map keyframes, leading to localization failure. In this regard, we propose a localization algorithm based on the global descriptor and dynamic range search (LA-GDADRS). In specific, we first design a global descriptor shift and rotation invariant image (SRI), which improves the rotation invariance and shift invariance by the methods of coordinates removal and de-centralization. Secondly, we design a global localization algorithm for shift and rotation invariant branch-and-bound scan matching (SRI-BBS). It first leverages SRI to obtain an approximate priori position of the unmanned vehicle and then utilizes the similarity between the current frame SRI and the map keyframes SRI to select a dynamic search range around the priori position. Within the search range, we leverage the branch-and-bound scanning matching algorithm to search for a more precise pose. It solves the problem that global localization tends to fail when the priori position is imprecise. Moreover, we introduce a tightly coupled factor graph model and a HD map engine to achieve real-time position tracking and lane-level localization, respectively. Finally, we complete extensive ablation experiments and comparative experiments to validate our methods on the benchmark dataset (KITTI) and the real application scenarios at the campus. Extensive experimental results demonstrate that our algorithm achieves the performance of mainstream localization algorithms. Full article

(This article belongs to the Topic Information Sensing Technology for Intelligent/Driverless Vehicle)

►▼ Show Figures

Graphical abstract

Graphical abstract
Full article ">Figure 1
<p>(<b>a</b>) Schematic diagram of LiDAR structure. (<b>b</b>) Schematic diagram of LiDAR measurement model.</p> Full article ">Figure 2
<p>Point cloud of LiDAR imaging results.</p> Full article ">Figure 3
<p>Overview of localization algorithm based on the global descriptor and dynamic range search.</p> Full article ">Figure 4
<p>Comparison after semantic segmentation and removal of dynamic objects: (<b>a</b>) point cloud after semantic segmentation, different colors are used to visualize the semantic information, the red points indicate vehicles, the green points indicate roads, the blue points indicate sidewalk, the purple points indicate buildings, and the light blue points indicate fence; (<b>b</b>) point cloud after removing dynamic objects.</p> Full article ">Figure 5
<p>Schematic diagram of global descriptor’s encoding process: the blue regions are element regions, the green lines are sectors, the purple circles are rings, and the other colored points are point clouds.</p> Full article ">Figure 6
<p>Flow chart of global localization.</p> Full article ">Figure 7
<p>Flow chart of rough localization.</p> Full article ">Figure 8
<p>Flow chart of fine localization.</p> Full article ">Figure 9
<p>Schematic diagram of the submap: each submap is 1/4 of the original. The rectangle filled with red in the submap represents the pixel of the submap and its value is 0 or 1.</p> Full article ">Figure 10
<p>Function images of similarity and search radius.</p> Full article ">Figure 11
<p>Flow chart of position tracking.</p> Full article ">Figure 12
<p>The structure of factor graph.</p> Full article ">Figure 13
<p>Grids for local map update: the yellow lines cut out the grids, the white points are the global map and the other colored points are the local map.</p> Full article ">Figure 14
<p>Schematic diagram of the lane model.</p> Full article ">Figure 15
<p>Display of data collection platforms: (<b>a</b>) J5 collection platform, (<b>b</b>) Avenger collection platform.</p> Full article ">Figure 16
<p>Sensor configurations and Volkswagen Tiguan data collection platform.</p> Full article ">Figure 17
<p>Schematics of point cloud’s global map (<b>a</b>) and testing data (<b>b</b>).</p> Full article ">Figure 18
<p>Comparison diagram of global localization failures: (<b>a</b>) shows the effect of SC-BBS, (<b>b</b>) shows the effect of our global localization algorithm. The green short lines indicate locations where localization is successful and the short red lines indicate locations where localization fails.</p> Full article ">Figure 18 Cont.
<p>Comparison diagram of global localization failures: (<b>a</b>) shows the effect of SC-BBS, (<b>b</b>) shows the effect of our global localization algorithm. The green short lines indicate locations where localization is successful and the short red lines indicate locations where localization fails.</p> Full article ">Figure 19
<p>Comparison of global localizations: (<b>a</b>) shows the comparison of global localization in the parking lot, (<b>b</b>) shows the comparison of global localization near the woods, (<b>c</b>) shows the comparison of global localization at the intersection. The area in the yellow box is the correct position of the unmanned vehicle, and the area in the red box is the wrong position of the unmanned vehicle.</p> Full article ">Figure 20
<p>Effects of global localization: (<b>a</b>) shows the global localization effect on a straight road, (<b>b</b>) shows the global localization effect near the woods, (<b>c</b>) shows the global localization effect in a parking lot, and (<b>d</b>) shows the global localization effect near a building. The white points are the global map, the blue points are the local map, and the other color points are the LiDAR scan.</p> Full article ">Figure 20 Cont.
<p>Effects of global localization: (<b>a</b>) shows the global localization effect on a straight road, (<b>b</b>) shows the global localization effect near the woods, (<b>c</b>) shows the global localization effect in a parking lot, and (<b>d</b>) shows the global localization effect near a building. The white points are the global map, the blue points are the local map, and the other color points are the LiDAR scan.</p> Full article ">Figure 21
<p>Relationship between success rates and distances to mapping trajectory.</p> Full article ">Figure 22
<p>Display of global localization on JLU campus dataset: the index (e.g., SRI: 275) in the figure represents the number of the most similar data frame retrieved.</p> Full article ">Figure 23
<p>Comparison between trajectories on JLU campus dataset.</p> Full article ">Figure 24
<p>Comparison between trajectories on KITTI dataset’s sequence 05.</p> Full article ">Figure 25
<p>The visualization of HD map localization: (<b>a</b>) shows the effect of HD map localization in front of a building, (<b>b</b>) shows the rear view of HD map localization, (<b>c</b>) shows the effect of HD map localization on a straight road, and (<b>d</b>) shows the effect of HD map localization near a wooded area. The blue polygons are the lanes, the light green polygon is the current lane, the white points are the global map, and the other color points are the point cloud collected by LiDAR.</p> Full article ">Figure A1
<p>Schematic diagram of sparse point–voxel convolution.</p> Full article ">

19 pages, 8819 KiB  

Open AccessArticle

Implementation Method of Automotive Video SAR (ViSAR) Based on Sub-Aperture Spectrum Fusion

by Ping Guo, Fuen Wu, Shiyang Tang, Chenghao Jiang and Changjie Liu

Remote Sens. 2023, 15(2), 476; https://doi.org/10.3390/rs15020476 - 13 Jan 2023

Cited by 6 | Viewed by 2264

Abstract

The automotive synthetic aperture radar (SAR) can obtain two-dimensional (2-D) high-resolution images and has good robustness compared with the other sensors. Generally, the 2-D high-resolution always conflicts with the real-time requirement in conventional SAR imaging. This article suggests an automotive video SAR (ViSAR) [...] Read more.

The automotive synthetic aperture radar (SAR) can obtain two-dimensional (2-D) high-resolution images and has good robustness compared with the other sensors. Generally, the 2-D high-resolution always conflicts with the real-time requirement in conventional SAR imaging. This article suggests an automotive video SAR (ViSAR) imaging technique based on sub-aperture spectrum fusion to address this issue. Firstly, the scene space variation problem caused by close observation distance in automotive SAR is analyzed. Moreover, the sub-aperture implementation method, frame rate and resolution of automotive ViSAR are also introduced. Then, the improved Range Doppler algorithm (RDA) is used to focus the sub-aperture data. Finally, a sub-aperture stitching strategy is proposed to obtain a high-resolution frame image. Compared with the available ViSAR imaging method, the proposed method is more efficient, performs better, and is more appropriate for automotive ViSAR. The simulation results and actual data of the automotive SAR validate the effectiveness of the proposed method. Full article

(This article belongs to the Topic Information Sensing Technology for Intelligent/Driverless Vehicle)

►▼ Show Figures

Graphical abstract

Graphical abstract
Full article ">Figure 1
<p>Automotive SAR geometric model.</p> Full article ">Figure 2
<p>Time-frequency characteristics of transmitted/received signals.</p> Full article ">Figure 3
<p>The phase error with range history simplification.</p> Full article ">Figure 4
<p>Schematic diagram of space-variant range curvature. (<b>a</b>) The phase error of space-variant curvature term; (<b>b</b>) Correction range curvature by an approximate method.</p> Full article ">Figure 5
<p>Implementation principle of automotive ViSAR with overlap.</p> Full article ">Figure 6
<p>Instantaneous Doppler frequency of a single-point target. (<b>a</b>) Coherent accumulation angle of sub-aperture; (<b>b</b>) Instantaneous Doppler bandwidth of sub-aperture.</p> Full article ">Figure 7
<p>Schematic diagram of sub-aperture data stitching.</p> Full article ">Figure 8
<p>Flowchart of the proposed method.</p> Full article ">Figure 9
<p>The ground scene for simulation.</p> Full article ">Figure 10
<p>ViSAR simulation result. (<b>a</b>) First frame image; (<b>b</b>) Second frame image; (<b>c</b>) Third frame image; (<b>d</b>) Fourth frame image; (<b>e</b>) Fifth frame image; (<b>f</b>) Sixth frame image.</p> Full article ">Figure 11
<p>Contour map of spectral fusion results for PT1. (<b>a</b>) Single sub-aperture fusion; (<b>b</b>) Two sub-aperture fusion; (<b>c</b>) Three sub-aperture fusion.</p> Full article ">Figure 12
<p>Comparative results of targets PT3, PT4, and PT5. (<b>a</b>) The traditional RDA; (<b>b</b>) The proposed method with segmented RCMC; (<b>c</b>) FFBPA.</p> Full article ">Figure 13
<p>Experimental system of automotive SAR.</p> Full article ">Figure 14
<p>Camera image of the experimental scene.</p> Full article ">Figure 15
<p>ViSAR imaging result of actual data. (<b>a</b>) First frame image; (<b>b</b>) Second frame image; (<b>c</b>) Third frame image; (<b>d</b>) Fourth frame image; (<b>e</b>) Fifth frame image; (<b>f</b>) Sixth frame image.</p> Full article ">Figure 16
<p>Comparative results of actual data. (<b>a</b>) The traditional RDA; (<b>b</b>) The proposed method with segmented RCMC; (<b>c</b>) FFBPA.</p> Full article ">Figure 17
<p>Schematic diagram of range cell migration.</p> Full article ">

26 pages, 39727 KiB  

Open AccessArticle

InTEn-LOAM: Intensity and Temporal Enhanced LiDAR Odometry and Mapping

by Shuaixin Li, Bin Tian, Xiaozhou Zhu, Jianjun Gui, Wen Yao and Guangyun Li

Remote Sens. 2023, 15(1), 242; https://doi.org/10.3390/rs15010242 - 31 Dec 2022

Cited by 4 | Viewed by 2461

Abstract

Traditional LiDAR odometry (LO) systems mainly leverage geometric information obtained from the traversed surroundings to register lazer scans and estimate LiDAR ego-motion, while they may be unreliable in dynamic or degraded environments. This paper proposes InTEn-LOAM, a low-drift and robust LiDAR odometry and [...] Read more.

Traditional LiDAR odometry (LO) systems mainly leverage geometric information obtained from the traversed surroundings to register lazer scans and estimate LiDAR ego-motion, while they may be unreliable in dynamic or degraded environments. This paper proposes InTEn-LOAM, a low-drift and robust LiDAR odometry and mapping method that fully exploits implicit information of lazer sweeps (i.e., geometric, intensity and temporal characteristics). The specific content of this work includes method innovation and experimental verification. With respect to method innovation, we propose the cylindrical-image-based feature extraction scheme, which makes use of the characteristic of uniform spatial distribution of lazer points to boost the adaptive extraction of various types of features, i.e., ground, beam, facade and reflector. We propose a novel intensity-based point registration algorithm and incorporate it into the LiDAR odometry, enabling the LO system to jointly estimate the LiDAR ego-motion using both geometric and intensity feature points. To eliminate the interference of dynamic objects, we propose a temporal-based dynamic object removal approach to filter them out in the resulting points map. Moreover, the local map is organized and downsampled using a temporal-related voxel grid filter to maintain the similarity between the current scan and the static local map. With respect to experimental verification, extensive tests are conducted on both simulated and real-world datasets. The results show that the proposed method achieves similar or better accuracy with respect to the state-of-the-art in normal driving scenarios and outperforms geometric-based LO in unstructured environments. Full article

(This article belongs to the Topic Information Sensing Technology for Intelligent/Driverless Vehicle)

►▼ Show Figures

Figure 1

Figure 1
<p>Overview of the proposed InTEn-LOAM system. (<b>a</b>) The color image from the on-board camera. (<b>b</b>) The projected scan-context segment image. (<b>c</b>) The raw point cloud from the on-board Velodyne HDL-64 LiDAR colored according to intensity. (<b>d</b>) The projected cylindrical range image colored according to depth. (<b>e</b>) The segmented label image. (<b>f</b>) The estimated normal image (<span style="color: #0000FF">x</span>, <span style="color: #00FF00">y</span>, <span style="color: #FF0000">z</span>). (<b>g</b>) The intensity image of non-ground points. Only reflector features are colored. (<b>h</b>) Various types of feature (<span style="color: #BDBDBD">ground</span>, <span style="color: #00FF00">facade</span>, <span style="color: #0000FF">beam</span>, <span style="color: #FA58AC">reflector</span>) extracted from the current lazer scan. (<b>i</b>) The current point features align with the local feature map that is in use so far (<span style="color: #FE9A2E">dynamic object in the current scan</span>).</p> Full article ">Figure 2
<p>Overall workflow of InTEn-LOAM.</p> Full article ">Figure 3
<p>The workflow of FEF.</p> Full article ">Figure 4
<p>A simple example of B-spline intensity surface model. The grid surface depicts the modeled continuous intensity surface with colors representing intensities and spheres in the center of surface grids representing control points of the B-spline surface model. <span style="color: #FF0000"><math display="inline"><semantics> <mrow> <msub> <mrow/> <mi>s</mi> </msub> <mi mathvariant="bold">p</mi> </mrow> </semantics></math></span> denotes the selected point and <span style="color: #0000FF"><math display="inline"><semantics> <mfenced separators="" open="{" close="}"> <msub> <mrow/> <mi>t</mi> </msub> <msub> <mi mathvariant="bold">q</mi> <mi>n</mi> </msub> </mfenced> </semantics></math></span> denotes query points. <span style="color: #FF0000"><math display="inline"><semantics> <mrow> <msub> <mrow/> <mi>s</mi> </msub> <mi mathvariant="bold">p</mi> </mrow> </semantics></math></span> is transformed to the reference frame of <span style="color: #0000FF"><math display="inline"><semantics> <mfenced separators="" open="{" close="}"> <msub> <mrow/> <mi>t</mi> </msub> <msub> <mi mathvariant="bold">q</mi> <mi>n</mi> </msub> </mfenced> </semantics></math></span> and denoted as <span style="color: #FA58AC"><math display="inline"><semantics> <mrow> <msub> <mrow/> <mi>t</mi> </msub> <mover accent="true"> <mi mathvariant="bold">p</mi> <mo>¯</mo> </mover> </mrow> </semantics></math></span>. <span style="color: #00FF00"><math display="inline"><semantics> <mrow> <msub> <mrow/> <mi>t</mi> </msub> <mi mathvariant="bold">q</mi> </mrow> </semantics></math></span> denotes the nearest neighboring query point of <span style="color: #FA58AC"><math display="inline"><semantics> <mrow> <msub> <mrow/> <mi>t</mi> </msub> <mover accent="true"> <mi mathvariant="bold">p</mi> <mo>¯</mo> </mover> </mrow> </semantics></math></span>.</p> Full article ">Figure 5
<p>The workflow of DOR.</p> Full article ">Figure 6
<p>Overview of four different types of feature associations. (<b>a</b>) Reflector; (<b>b</b>) Facade; (<b>c</b>) Edge; (<b>d</b>) Ground feature association.</p> Full article ">Figure 7
<p>Dataset sampling platform. (<b>a</b>) Autonomous driving car; (<b>b</b>) Simulated mine car and scan example. <span style="color: #FA58AC">Magenta</span> lazer points are reflected from <span style="color: #F3E2A9">brown</span> signs in the simulated environment.</p> Full article ">Figure 8
<p>Feature extraction results in different scenes. (<b>a</b>) Open road; (<b>b</b>) City avenue; (<b>c</b>) Long straight tunnel; (<b>d</b>) Roadside green belt. (<span style="color: #00FF00">plane</span>, <span style="color: #FF0000">reflector</span>, <span style="color: #0000FF">edge</span> and <span style="color: #BDBDBD">raw scan</span> points). Objects in the real-world scenes and their counterparts in lazer scans are indicated by boxes (<span style="color: #FF0000">reflector features</span>, <span style="color: #0000FF">edge features</span>, <span style="color: #FFBF00">some special areas</span>).</p> Full article ">Figure 9
<p>Relative error plots. (<b>a</b>) Relative translation error curves; (<b>b</b>) Relative rotation error curves. (<span style="color: #00FF00">NDT</span> of HDL-Graph-SLAM, <span style="color: #FA58AC">feature-based registration</span> approach of LOAM, the proposed <span style="color: #0000FF">intensity-based registration</span> approach).</p> Full article ">Figure 10
<p>DOR examples for a single frame of lazer scan. (<b>a</b>) Seq.07. Vehicles crossing the intersection when the data collection vehicle stops and waits for the traffic light (<b>left</b>); The cyclist traveling in the opposite direction when the data collection vehicle is driving along the road (<b>right</b>). (<b>b</b>) Seq.10. Followers behind the data collection vehicle as it travels down the highway at high speed (<b>top</b>); Vehicles driving in the opposite direction and in front of the data collection vehicle when it slows down (<b>bottom</b>). (<span style="color: #00FF00">facade</span>, <span style="color: #BDBDBD">ground</span>, <span style="color: #0000FF">edge</span> and <span style="color: #FFBF00">dynamics</span> for points <span style="color: #FFBF00">true positive</span>, <span style="color: #FF0000">false positive</span> and <span style="color: #00FF00">true negative</span> for dynamic segmentation boxes.)</p> Full article ">Figure 11
<p>Comparison between local maps of LOAM and InTEn-LOAM. (<b>a</b>) Map at the intersection; (<b>b</b>) Map at the busy road. In each subfigure, the top represents the map of LOAM without DOR, while the bottom represents the map of InTEn-LOAM with DOR.</p> Full article ">Figure 12
<p>The average RTE and RRE of InTEn-LOAM over fixed lengths. (<b>a</b>) RTE; (<b>b</b>) RRE.</p> Full article ">Figure 13
<p>Constructed points maps with details and estimated trajectories. (<b>a</b>,<b>c</b>,<b>e</b>) maps of Seq.00, 01 and 05; (<b>b</b>,<b>d</b>,<b>f</b>) trajectories of Seq.00, 01 and 05 (<span style="color: #FA58AC">groundtruths</span> and <span style="color: #0000FF">InTEn-LOAM</span>).</p> Full article ">Figure 14
<p>Cumulative distributions of absolute state errors and estimated trajectories. (<b>a</b>) Cumulative distributions of the absolute positioning errors; (<b>b</b>) Cumulative distributions of the absolute rotational errors; (<b>c</b>) Estimated trajectories. (<span style="color: #0000FF">InTEn-LOAM</span>, <span style="color: #FA58AC">LOAM</span>, <span style="color: #00FF00">HDL-Graph-SLAM</span>, <span style="color: #FFBF00">groundtruth</span>).</p> Full article ">Figure 15
<p>The average RTE and RRE of LO systems over fixed lengths. (<b>a</b>) RTE; (<b>b</b>) RRE.</p> Full article ">Figure 16
<p>InTEn-LOAM’s map result on urban scenario (KITTI seq.06): (<b>a</b>) overview, (<b>b</b>) map in detail of circled areas, (<b>c</b>) reference map comparison.</p> Full article ">Figure 17
<p>InTEn-LOAM’s map result on country scenario (KITTI seq.10): (<b>a</b>) overview, (<b>b</b>) map in detail of circled areas, (<b>c</b>) reference map comparison.</p> Full article ">Figure 18
<p>LO systems’ map results on autonomous driving field dataset in the tunnel region. (<b>a</b>) InTEn-LOAM, (<b>b</b>) LOAM, (<b>c</b>) HDL-Graph-SLAM, (<b>d</b>) MULLS.</p> Full article ">Figure 19
<p>InTEn-LOAM’s map result on autonomous driving field dataset. (<b>a</b>) the constructed point cloud map, (<b>b</b>) local remote sensing image and estimated trajectory.</p> Full article ">Figure 19 Cont.
<p>InTEn-LOAM’s map result on autonomous driving field dataset. (<b>a</b>) the constructed point cloud map, (<b>b</b>) local remote sensing image and estimated trajectory.</p> Full article ">

19 pages, 6550 KiB  

Open AccessArticle

Anti-Noise 3D Object Detection of Multimodal Feature Attention Fusion Based on PV-RCNN

by Yuan Zhu, Ruidong Xu, Hao An, Chongben Tao and Ke Lu

Sensors 2023, 23(1), 233; https://doi.org/10.3390/s23010233 - 26 Dec 2022

Cited by 5 | Viewed by 3153

Abstract

3D object detection methods based on camera and LiDAR fusion are susceptible to environmental noise. Due to the mismatch of physical characteristics of the two sensors, the feature vectors encoded by the feature layer are in different feature spaces. This leads to the [...] Read more.

3D object detection methods based on camera and LiDAR fusion are susceptible to environmental noise. Due to the mismatch of physical characteristics of the two sensors, the feature vectors encoded by the feature layer are in different feature spaces. This leads to the problem of feature information deviation, which has an impact on detection performance. To address this problem, a point-guided feature abstract method is presented to fuse the camera and LiDAR at first. The extracted image features and point cloud features are aggregated to keypoints for enhancing information redundancy. Second, the proposed multimodal feature attention (MFA) mechanism is used to achieve adaptive fusion of point cloud features and image features with information from multiple feature spaces. Finally, a projection-based farthest point sampling (P-FPS) is proposed to downsample the raw point cloud, which can project more keypoints onto the close object and improve the sampling rate of the point-guided image features. The 3D bounding boxes of the object is obtained by the region of interest (ROI) pooling layer and the fully connected layer. The proposed 3D object detection algorithm is evaluated on three different datasets, and the proposed algorithm achieved better detection performance and robustness when the image and point cloud data contain rain noise. The test results on a physical test platform further validate the effectiveness of the algorithm. Full article

(This article belongs to the Topic Information Sensing Technology for Intelligent/Driverless Vehicle)

►▼ Show Figures

Figure 1

Figure 1
<p>The architecture of the proposed 3D object detection.</p> Full article ">Figure 2
<p>Comparison of (<b>a</b>) FPS and (<b>b</b>) P-FPS.</p> Full article ">Figure 3
<p>Illustration of the P-FPS.</p> Full article ">Figure 4
<p>Multimodal Feature Attention (MFA) fusion module.</p> Full article ">Figure 5
<p>Region of Interest (ROI) pooling module and detection head.</p> Full article ">Figure 6
<p>The effect of adding rain noise to the data.</p> Full article ">Figure 7
<p>Precision–recall curve.</p> Full article ">Figure 8
<p>Physical test platform.</p> Full article ">Figure 9
<p>Visual results of physical test platform dataset.</p> Full article ">Figure 10
<p>Comparison of P-FPS and FPS on recalling GT points: (<b>a</b>) P-FPS (recall 1507 points); (<b>b</b>) FPS (recall 575 points).</p> Full article ">Figure 11
<p>Performance comparisons on different <math display="inline"><semantics> <mrow> <mi mathvariant="sans-serif">α</mi> </mrow> </semantics></math>.</p> Full article ">Figure 12
<p>Effect of fusion feature on Predicted Keypoint Weighting (PKW). (<b>a</b>) Foreground point prediction with fusion feature. (<b>b</b>) Foreground point prediction without fusion feature.</p> Full article ">Figure 13
<p>Performance comparison on the car category with different heads.</p> Full article ">Figure 14
<p>Visualization of MFA. Green/blue/yellow boxes indicate Car/Pedestrian/Cyclist.</p> Full article ">

24 pages, 10865 KiB  

Open AccessArticle

LiDAR Odometry and Mapping Based on Neighborhood Information Constraints for Rugged Terrain

by Gang Wang, Xinyu Gao, Tongzhou Zhang, Qian Xu and Wei Zhou

Remote Sens. 2022, 14(20), 5229; https://doi.org/10.3390/rs14205229 - 19 Oct 2022

Cited by 3 | Viewed by 2155

Abstract

The simultaneous localization and mapping (SLAM) method estimates vehicles’ pose and builds maps established on the collection of environmental information primarily through sensors such as LiDAR and cameras. Compared to the camera-based SLAM, the LiDAR-based SLAM is more geared to complicated environments and [...] Read more.

The simultaneous localization and mapping (SLAM) method estimates vehicles’ pose and builds maps established on the collection of environmental information primarily through sensors such as LiDAR and cameras. Compared to the camera-based SLAM, the LiDAR-based SLAM is more geared to complicated environments and is not susceptible to weather and illumination, which has increasingly become a hot topic in autonomous driving. However, there has been relatively little research on the LiDAR-based SLAM algorithm in rugged scenes. The following two issues remain unsolved: on the one hand, the small overlap area of two adjacent point clouds results in insufficient valuable features that can be extracted; on the other hand, the conventional feature matching method does not take point cloud pitching into account, which frequently results in matching failure. Hence, a LiDAR SLAM algorithm based on neighborhood information constraints (LoNiC) for rugged terrain is proposed in this study. Firstly, we obtain the feature points with surface information using the distribution of the normal vector angles in the neighborhood and extract features with discrimination through the local surface information of the point cloud, to improve the describing ability of feature points in rugged scenes. Secondly, we provide a multi-scale constraint description based on point cloud curvature, normal vector angle, and Euclidean distance to enhance the algorithm’s discrimination of the differences between feature points and prevent mis-registration. Subsequently, in order to lessen the impact of the initial pose value on the precision of point cloud registration, we introduce the dynamic iteration factor to the registration process and modify the corresponding relationship of the matching point pairs by adjusting the distance and angle thresholds. Finally, the verification based on the KITTI and JLU campus datasets verifies that the proposed algorithm significantly improves the accuracy of mapping. Specifically in rugged scenes, the mean relative translation error is 0.0173%, and the mean relative rotation error is 2.8744°/m, reaching the current level of the state of the art (SOTA) method. Full article

(This article belongs to the Topic Information Sensing Technology for Intelligent/Driverless Vehicle)

►▼ Show Figures

Graphical abstract

Graphical abstract
Full article ">Figure 1
<p>The schematic diagram of SLAM method in unmanned navigation system.</p> Full article ">Figure 2
<p>Data collection platforms from JLUROBOT team: (<b>a</b>) J5; (<b>b</b>) Avenger.</p> Full article ">Figure 3
<p>Algorithm overview.</p> Full article ">Figure 4
<p>Feature described by surface information extraction.</p> Full article ">Figure 5
<p>Surface fluctuation of local neighborhood. The red circle represents the current point cloud<math display="inline"><semantics> <mrow> <mo> </mo> <msub> <mi>p</mi> <mi>i</mi> </msub> </mrow> </semantics></math>, and the blue circles represent the <span class="html-italic">k</span> neighbors around<math display="inline"><semantics> <mrow> <mo> </mo> <msub> <mi>p</mi> <mi>i</mi> </msub> </mrow> </semantics></math>: (<b>a</b>) When <math display="inline"><semantics> <mrow> <msub> <mi>λ</mi> <mrow> <mi>i</mi> <mn>1</mn> </mrow> </msub> <mo>=</mo> <msub> <mi>λ</mi> <mrow> <mi>i</mi> <mn>2</mn> </mrow> </msub> <mo>=</mo> <msub> <mi>λ</mi> <mrow> <mi>i</mi> <mn>3</mn> </mrow> </msub> </mrow> </semantics></math>, the surface fluctuates the most; (<b>b</b>) when <math display="inline"><semantics> <mrow> <msub> <mi>λ</mi> <mrow> <mi>i</mi> <mn>1</mn> </mrow> </msub> <mo>≪</mo> <mo> </mo> <msub> <mi>λ</mi> <mrow> <mi>i</mi> <mn>2</mn> </mrow> </msub> <mo>,</mo> <mo> </mo> <msub> <mi>λ</mi> <mrow> <mi>i</mi> <mn>1</mn> </mrow> </msub> <mo>≪</mo> <mo> </mo> <msub> <mi>λ</mi> <mrow> <mi>i</mi> <mn>3</mn> </mrow> </msub> </mrow> </semantics></math>, the surface is close to a plane.</p> Full article ">Figure 6
<p>Smoothness of the region: (<b>a</b>) undulating region; (<b>b</b>) smooth region.</p> Full article ">Figure 7
<p>Multi-scale constraint description matching search.</p> Full article ">Figure 8
<p>Registration with dynamic iteration factors.</p> Full article ">Figure 9
<p>Point cloud iterative matching process: <math display="inline"><semantics> <mrow> <mi>Q</mi> <mo> </mo> </mrow> </semantics></math>is iterated to position <math display="inline"><semantics> <msup> <mi>Q</mi> <mo>′</mo> </msup> </semantics></math> with a smaller offset angle and distance from <math display="inline"><semantics> <mi>P</mi> </semantics></math>.</p> Full article ">Figure 10
<p>Mobile platform equipped with sensors (Volkswagen Tiguan).</p> Full article ">Figure 11
<p>Trajectory comparison of sequence 01 on the KITTI dataset.</p> Full article ">Figure 12
<p>Trajectory comparison of sequence 09 on the KITTI dataset.</p> Full article ">Figure 13
<p>Mapping trajectory: (<b>a</b>) mapping trajectory of sequence 05 on the KITTI dataset; (<b>b</b>) mapping trajectory of sequence 07 on the KITTI dataset.</p> Full article ">Figure 14
<p>Trajectory comparison: (<b>a</b>) trajectory comparison of sequence 05 on the KITTI dataset; (<b>b</b>) trajectory comparison of sequence 07 on the KITTI dataset.</p> Full article ">Figure 14 Cont.
<p>Trajectory comparison: (<b>a</b>) trajectory comparison of sequence 05 on the KITTI dataset; (<b>b</b>) trajectory comparison of sequence 07 on the KITTI dataset.</p> Full article ">Figure 15
<p>Trajectory map of sequence JLU_070101 on the JLU campus dataset: a, b and c refer to the comparison of elevation fluctuation trend, mapping trajectory, and mapping effect of road sections at the marked positions of (<b>a</b>–<b>c</b>), respectively.</p> Full article ">Figure 16
<p>BEV image of the JLU_062801 sequence, LoNiC mapping results, and the superimposed renderings of the BEV image and the point cloud image (the colored trajectory in the BEV image refers to the actual trajectory of the Volkswagen Tiguan).</p> Full article ">Figure 17
<p>BEV image of the JLU_070102 sequence, LoNiC mapping results, and the superimposed renderings of the BEV image and the point cloud image (the colored trajectory in the BEV image refers to the actual trajectory of the Volkswagen Tiguan).</p> Full article ">

16 pages, 4339 KiB  

Open AccessArticle

An Innovative and Cost-Effective Traffic Information Collection Scheme Using the Wireless Sniffing Technique

by Wei-Hsun Lee, Teng-Jyun Liang and Hsuan-Chih Wang

Vehicles 2022, 4(4), 996-1011; https://doi.org/10.3390/vehicles4040054 - 30 Sep 2022

Cited by 1 | Viewed by 2029

Abstract

In recent years, the wireless sniffing technique (WST) has become an emerging technique for collecting real-time traffic information. The spatiotemporal variations in wireless signal collection from vehicles provide various types of traffic information, such as travel time, speed, traveling path, and vehicle turning [...] Read more.

In recent years, the wireless sniffing technique (WST) has become an emerging technique for collecting real-time traffic information. The spatiotemporal variations in wireless signal collection from vehicles provide various types of traffic information, such as travel time, speed, traveling path, and vehicle turning proportion at an intersection, which can be widely used for traffic management applications. However, three problems challenge the applicability of the WST to traffic information collection: the transportation mode classification problem (TMP), lane identification problem (LIP), and multiple devices problem (MDP). In this paper, a WST-based intelligent traffic beacon (ITB) with machine learning methods, including SVM, KNN, and AP, is designed to solve these problems. Several field experiments are conducted to validate the proposed system: three sensor topologies (X-type, rectangle-type, and diamond-type topologies) with two wireless sniffing schemes (Bluetooth and Wi-Fi). Experiment results show that X-type has the best performance among all topologies. For sniffing schemes, Bluetooth outperforms Wi-Fi. With the proposed ITB solution, traffic information can be collected in a more cost-effective way. Full article

(This article belongs to the Topic Information Sensing Technology for Intelligent/Driverless Vehicle)

►▼ Show Figures

Figure 1

Figure 1
<p>The Z-type topology (Fan [<a href="#B6-vehicles-04-00054" class="html-bibr">6</a>]).</p> Full article ">Figure 2
<p>The topologies of ITB: (<b>a</b>) X-type; (<b>b</b>) rectangle-type; (<b>c</b>) diamond-type.</p> Full article ">Figure 3
<p>Research scenario.</p> Full article ">Figure 4
<p>The variation of RSSI in TMP.</p> Full article ">Figure 5
<p>Variation of RSSI for LIP in two cases. (number 1~6 indicates the ITB no. shown in <a href="#vehicles-04-00054-f003" class="html-fig">Figure 3</a>) (<b>a</b>) Driving without changing lane. (<b>b</b>) Driving and changing lane.</p> Full article ">Figure 6
<p>RSSI of MDP. (<b>a</b>) Mobile devices in different vehicles. (<b>b</b>) Multiple devices in the same vehicle.</p> Full article ">Figure 7
<p>Framework for the collection of traffic data by the WST.</p> Full article ">Figure 8
<p>An observation of collected Wi-Fi signal raw data of six smartphones.</p> Full article ">Figure 9
<p>Experiment field of TMP (in NCKU campus).</p> Full article ">Figure 10
<p>Confusion matrixes of three topologies for BT.</p> Full article ">Figure 11
<p>Confusion matrixes of three topologies for Wi-Fi.</p> Full article ">Figure 12
<p>TMP accuracy.</p> Full article ">Figure 13
<p>LIP experiment field (in Annan district, Tainan City, Taiwan).</p> Full article ">Figure 14
<p>The hierarchical SVM structure proposed for solving LIP.</p> Full article ">Figure 15
<p>Accuracy of LIP comparison on different topologies: (<b>a</b>) accuracy of three SVMs on Bluetooth; (<b>b</b>) accuracy of three SVMs on Wi-Fi.</p> Full article ">Figure 16
<p>(<b>a</b>) Scenario 1: the vehicles are in parallel; (<b>b</b>) Scenario 2: the vehicles are in tandem.</p> Full article ">Figure 17
<p>Accuracy of MDP comparison on different topologies: (<b>a</b>) accuracy of MDP in Bluetooth; (<b>b</b>) accuracy of MDP in Wi-Fi.</p> Full article ">

18 pages, 10057 KiB  

Open AccessArticle

A Coupled Visual and Inertial Measurement Units Method for Locating and Mapping in Coal Mine Tunnel

by Daixian Zhu, Kangkang Ji, Dong Wu and Shulin Liu

Sensors 2022, 22(19), 7437; https://doi.org/10.3390/s22197437 - 30 Sep 2022

Cited by 5 | Viewed by 2482

Abstract

Mobile robots moving fast or in scenes with poor lighting conditions often cause the loss of visual feature tracking. In coal mine tunnels, the ground is often bumpy and the lighting is uneven. During the movement of the mobile robot in this scene, [...] Read more.

Mobile robots moving fast or in scenes with poor lighting conditions often cause the loss of visual feature tracking. In coal mine tunnels, the ground is often bumpy and the lighting is uneven. During the movement of the mobile robot in this scene, there will be violent bumps. The localization technology through visual features is greatly affected by the illumination and the speed of the camera movement. To solve the localization and mapping problem in an environment similar to underground coal mine tunnels, we improve a localization and mapping algorithm based on a monocular camera and an Inertial Measurement Unit (IMU). A feature-matching method that combines point and line features is designed to improve the robustness of the algorithm in the presence of degraded scene structure and insufficient illumination. The tightly coupled method is used to establish visual feature constraints and IMU pre-integration constraints. A keyframe nonlinear optimization algorithm based on sliding windows is used to accomplish state estimation. Extensive simulations and practical environment verification show that the improved simultaneous localization and mapping (SLAM) system with a monocular camera and IMU fusion can achieve accurate autonomous localization and map construction in scenes with insufficient light such as coal mine tunnels. Full article

(This article belongs to the Topic Information Sensing Technology for Intelligent/Driverless Vehicle)

►▼ Show Figures

Figure 1

Figure 1
<p>System block diagram.</p> Full article ">Figure 2
<p>IMU pre-integration.</p> Full article ">Figure 3
<p>Image frame initialization.</p> Full article ">Figure 4
<p>Reprojection error of point and line features.</p> Full article ">Figure 5
<p>Data association.</p> Full article ">Figure 6
<p>Closed-loop detection.</p> Full article ">Figure 7
<p>Results of feature point extraction and matching.</p> Full article ">Figure 8
<p>Results of feature line extraction and matching.</p> Full article ">Figure 9
<p>Image with insufficient illumination.</p> Full article ">Figure 10
<p>Image with insufficient illumination.</p> Full article ">Figure 11
<p>Partial dataset image.</p> Full article ">Figure 12
<p>Comparison of trajectory error.</p> Full article ">Figure 13
<p>Scenes from coal mine tunnel.</p> Full article ">Figure 14
<p>Trajectory error between the two algorithms and the true value under two paths.</p> Full article ">

17 pages, 4788 KiB  

Open AccessArticle

Unifying Deep ConvNet and Semantic Edge Features for Loop Closure Detection

by Jie Jin, Jiale Bai, Yan Xu and Jiani Huang

Remote Sens. 2022, 14(19), 4885; https://doi.org/10.3390/rs14194885 - 30 Sep 2022

Cited by 2 | Viewed by 1750

Abstract

Loop closure detection is an important component of Simultaneous Localization and Mapping (SLAM). In this paper, a novel two-branch loop closure detection algorithm unifying deep Convolutional Neural Network (ConvNet) features and semantic edge features is proposed. In detail, we use one feature extraction [...] Read more.

Loop closure detection is an important component of Simultaneous Localization and Mapping (SLAM). In this paper, a novel two-branch loop closure detection algorithm unifying deep Convolutional Neural Network (ConvNet) features and semantic edge features is proposed. In detail, we use one feature extraction module to extract both ConvNet and semantic edge features simultaneously. The deep ConvNet features are subjected to a Context Feature Enhancement (CFE) module in the global feature ranking branch to generate a representative global feature descriptor. Concurrently, to reduce the interference of dynamic features, the extracted semantic edge information of landmarks is encoded through the Vector of Locally Aggregated Descriptors (VLAD) framework in the semantic edge feature ranking branch to form semantic edge descriptors. Finally, semantic, visual, and geometric information is integrated by the similarity score fusion calculation. Extensive experiments on six public datasets show that the proposed approach can achieve competitive recall rates at 100% precision compared to other state-of-the-art methods. Full article

(This article belongs to the Topic Information Sensing Technology for Intelligent/Driverless Vehicle)

►▼ Show Figures

Graphical abstract

Graphical abstract
Full article ">Figure 1
<p>An overview of the proposed module. As the incoming image stream enters the pipeline, the ConvNet features and the semantic edge features of the image are extracted by the feature extraction module. The first ones enter the global feature ranking branch to retrieve the most similar ConvNet candidates. The semantic edges are sent to the semantic edge feature ranking branch to select the most similar images in human vision. Finally, the matched image pair is generated by similarity score fusion computation.</p> Full article ">Figure 2
<p>The overall architecture of our proposed Context Feature Enhanced (CFE) module.</p> Full article ">Figure 3
<p>Visualization of semantic edge features. (<b>a</b>) raw image. (<b>b</b>) semantic edge images of all categories (<b>above</b>), dynamic categories (<b>bottom left</b>) and static categories (<b>bottom right</b>).</p> Full article ">Figure 3 Cont.
<p>Visualization of semantic edge features. (<b>a</b>) raw image. (<b>b</b>) semantic edge images of all categories (<b>above</b>), dynamic categories (<b>bottom left</b>) and static categories (<b>bottom right</b>).</p> Full article ">Figure 4
<p>Effect of semantic categories on six datasets.</p> Full article ">Figure 5
<p>The Precision-Recall curves of the two branches on six datasets. (<b>a</b>) Global feature ranking branch. (<b>b</b>) Semantic edge feature ranking branch.</p> Full article ">Figure 6
<p>Effect of weighted fusion on KITTI00, KITTI05, Malaga#8 datasets.</p> Full article ">Figure 7
<p>Trajectories of the six datasets and all detected loop closures and false negative loop closures at 100% precision. (<b>a</b>) Results of the proposed method on KITTI00. (<b>b</b>) Results of the proposed method on KITTI05. (<b>c</b>) Results of the proposed method on KITTI06. (<b>d</b>) Results of the proposed method on KITTI09. (<b>e</b>) Results of the proposed method on Malaga#8. (<b>f</b>) Results of the proposed method on CC.</p> Full article ">Figure 7 Cont.
<p>Trajectories of the six datasets and all detected loop closures and false negative loop closures at 100% precision. (<b>a</b>) Results of the proposed method on KITTI00. (<b>b</b>) Results of the proposed method on KITTI05. (<b>c</b>) Results of the proposed method on KITTI06. (<b>d</b>) Results of the proposed method on KITTI09. (<b>e</b>) Results of the proposed method on Malaga#8. (<b>f</b>) Results of the proposed method on CC.</p> Full article ">

24 pages, 7357 KiB  

Open AccessArticle

Where Am I? SLAM for Mobile Machines on a Smart Working Site

by Yusheng Xiang, Dianzhao Li, Tianqing Su, Quan Zhou, Christine Brach, Samuel S. Mao and Marcus Geimer

Vehicles 2022, 4(2), 529-552; https://doi.org/10.3390/vehicles4020031 - 27 May 2022

Cited by 3 | Viewed by 2529

Abstract

The current optimization approaches of construction machinery are mainly based on internal sensors. However, the decision of a reasonable strategy is not only determined by its intrinsic signals, but also very strongly by environmental information, especially the terrain. Due to the dynamic changing [...] Read more.

The current optimization approaches of construction machinery are mainly based on internal sensors. However, the decision of a reasonable strategy is not only determined by its intrinsic signals, but also very strongly by environmental information, especially the terrain. Due to the dynamic changing of the construction site and the consequent absence of a high definition map, the Simultaneous Localization and Mapping (SLAM) offering the terrain information for construction machines is still challenging. Current SLAM technologies proposed for mobile machines are strongly dependent on costly or computationally expensive sensors, such as RTK GPS and cameras, so that commercial use is rare. In this study, we proposed an affordable SLAM method to create a multi-layer grid map for the construction site so that the machine can have the environmental information and be optimized accordingly. Concretely, after the machine passes by the grid, we can obtain the local information and record it. Combining with positioning technology, we then create a map of the interesting places of the construction site. As a result of our research gathered from Gazebo, we showed that a suitable layout is the combination of one IMU and two differential GPS antennas using the unscented Kalman filter, which keeps the average distance error lower than 2m and the mapping error lower than 1.3% in the harsh environment. As an outlook, our SLAM technology provides the cornerstone to activate many efficiency improvement approaches. Full article

(This article belongs to the Topic Information Sensing Technology for Intelligent/Driverless Vehicle)

►▼ Show Figures

Figure 1

Figure 1
<p>Environment based optimization. Mobile machines perform tasks more efficiently or safer according to their location and surroundings information. The short-term goal of SLAM is to prevent construction machinery from always working in low-efficiency areas for safety reasons, whereas the long-term goal is to increase the productivity of the working site with the help of path planning. The study focuses on affordable SLAM technology for construction machines. This work is also presented in my thesis [<a href="#B14-vehicles-04-00031" class="html-bibr">14</a>].</p> Full article ">Figure 2
<p>Wheel loader model in Gazebo: once the models had been developed in Solidwork, they were converted to Unified Robotic Description Format (URDF), using a 3rd party URDF conversion tool called “<span class="html-italic">sw_urdf_exporter</span>”, which allows for conveniently exporting SW Parts and Assemblies into a URDF file. Gazebo enables us to obtain sensors’ simulation such as IMUs, GPSs, encoders, cameras, and stereo cameras through <span class="html-italic">gazebo_plugins</span>, which can be used to attach into ROS messages and service calling the sensor outputs, i.e., the <span class="html-italic">gazebo_plugins</span> create a complete interface (Topic) between ROS and Gazebo.</p> Full article ">Figure 3
<p>The dynamic system simulated by URDF file on ROS.</p> Full article ">Figure 4
<p>Our approach uses multilayered grid maps to store data for different types of information. Concretely, every grid saves a <math display="inline"><semantics> <mrow> <mn>1</mn> <mo>×</mo> <mn>3</mn> </mrow> </semantics></math> matrix including location information and resistance or grade, depending on which layer it is. A grid with site information will be created after the vehicle passes by. The map is saved as a <math display="inline"><semantics> <mrow> <mn>2</mn> <mo>×</mo> <mi>m</mi> <mo>×</mo> <mi>n</mi> <mo>×</mo> <mn>3</mn> </mrow> </semantics></math> tensor, where m is the max displacement in the x-direction, while n is the max displacement in the y-direction. In case a grid is not occupied once by the vehicle, it will be marked as NaN to denote the unknown regions.</p> Full article ">Figure 5
<p>Detail description of a layer in the grid-based map.</p> Full article ">Figure 6
<p>The ground truth map with dimensions. The simulation environment we used in Gazebo was modeled based on a real construction site, and the parameters are selected according to material characteristics. Since simulating a small construction site may cause system error and thus lack plausibility, we augmented this real construction site’s dimensions in Gazebo. (<b>a</b>) Example construction site divided into five areas according to different ground resistances; (<b>b</b>) Example construction site divided into two areas according to different slopes.</p> Full article ">Figure 7
<p>The IMU + Odometry estimation method yields for both EKF and UKF an RMSE over 70 m. Except for the IMU + Odometry methods, the other results can be divided into four performance levels according to the error scale. The worst level is EKF with one GPS, where the RMSE is about <math display="inline"><semantics> <mrow> <mn>3</mn> <mo>∼</mo> <mn>4</mn> </mrow> </semantics></math> m. The second level is EKF with two GPS and EKF with three IMU and three GPS. In these cases, the RMSEs are about <math display="inline"><semantics> <mrow> <mn>2</mn> <mo>∼</mo> <mn>3</mn> </mrow> </semantics></math> m. A better level is UKF with 1 GPS and 1 IMU, where the RMSE of the third level can be achieved about <math display="inline"><semantics> <mrow> <mn>2</mn> <mo>∼</mo> <mn>2.5</mn> </mrow> </semantics></math> m. Finally, our experiment’s best class is UKF with 2 GPS and 1 IMU, and UKF with 3 IMU and 3 GPS, which reduce the RMSE to about 1.2 m. (<b>a</b>) EKF with only one IMU. Here the red line is the ground truth and the blue line denotes the estimated position. This definition also applies to later subfigures; (<b>b</b>) UKF with only one IMU; (<b>c</b>) EKF with 1 IMU and 1 GPS; (<b>d</b>) UKF with 1 IMU and 1 GPS; (<b>e</b>) EKF with 2 IMU and 1 GPS; (<b>f</b>) UKF with 2 IMU and 1 GPS; (<b>g</b>) EKF with 3 IMU and 1 GPS; (<b>h</b>) UKF with 3 IMU and 1 GPS; (<b>i</b>) EKF with 1 IMU and 2 GPS; (<b>j</b>) UKF with 1 IMU and 2 GPS; (<b>k</b>) EKF with 2 IMU and 2 GPS; (<b>l</b>) UKF with 2 IMU and 2 GPS; (<b>m</b>) EKF with 3 IMU and 2 GPS; (<b>n</b>) UKF with 3 IMU and 2 GPS; (<b>o</b>) EKF with 3 IMU and 3 GPS; (<b>p</b>) UKF with 3 IMU and 3 GPS.</p> Full article ">Figure 7 Cont.
<p>The IMU + Odometry estimation method yields for both EKF and UKF an RMSE over 70 m. Except for the IMU + Odometry methods, the other results can be divided into four performance levels according to the error scale. The worst level is EKF with one GPS, where the RMSE is about <math display="inline"><semantics> <mrow> <mn>3</mn> <mo>∼</mo> <mn>4</mn> </mrow> </semantics></math> m. The second level is EKF with two GPS and EKF with three IMU and three GPS. In these cases, the RMSEs are about <math display="inline"><semantics> <mrow> <mn>2</mn> <mo>∼</mo> <mn>3</mn> </mrow> </semantics></math> m. A better level is UKF with 1 GPS and 1 IMU, where the RMSE of the third level can be achieved about <math display="inline"><semantics> <mrow> <mn>2</mn> <mo>∼</mo> <mn>2.5</mn> </mrow> </semantics></math> m. Finally, our experiment’s best class is UKF with 2 GPS and 1 IMU, and UKF with 3 IMU and 3 GPS, which reduce the RMSE to about 1.2 m. (<b>a</b>) EKF with only one IMU. Here the red line is the ground truth and the blue line denotes the estimated position. This definition also applies to later subfigures; (<b>b</b>) UKF with only one IMU; (<b>c</b>) EKF with 1 IMU and 1 GPS; (<b>d</b>) UKF with 1 IMU and 1 GPS; (<b>e</b>) EKF with 2 IMU and 1 GPS; (<b>f</b>) UKF with 2 IMU and 1 GPS; (<b>g</b>) EKF with 3 IMU and 1 GPS; (<b>h</b>) UKF with 3 IMU and 1 GPS; (<b>i</b>) EKF with 1 IMU and 2 GPS; (<b>j</b>) UKF with 1 IMU and 2 GPS; (<b>k</b>) EKF with 2 IMU and 2 GPS; (<b>l</b>) UKF with 2 IMU and 2 GPS; (<b>m</b>) EKF with 3 IMU and 2 GPS; (<b>n</b>) UKF with 3 IMU and 2 GPS; (<b>o</b>) EKF with 3 IMU and 3 GPS; (<b>p</b>) UKF with 3 IMU and 3 GPS.</p> Full article ">Figure 8
<p>Quantitative evaluation of different methods. Here (<b>a</b>) RMSE of EKF and (<b>b</b>) RMSE of UKF show the accumulative error, i.e., RMSE, while (<b>c</b>) Euclidean distance error of EKF and (<b>d</b>) Euclidean distance error of UKF demonstrate the current Euclidean distance error of each sensor arrangement. There are some noticeable instantaneous position changes every ten seconds due to infrequent GPS signal loss. Generally, UKF shows better performance than EKF for both RMSE and Euclidean distance error.</p> Full article ">Figure 9
<p>The ground truth and estimated maps. In the ground resistance map and road grade map plotted by EKF with 1 IMU and 1 GPS, the spikes caused by infrequent GPS are quite obvious. With an additional GPS sensor fused in Kalman filter, the spikes improve a lot. (<b>a</b>) Predefined five areas of resistance plotted by OpenCV; (<b>b</b>) Result of ground resistance map with wheel loader (EKF with 1 IMU and 1 GPS); (<b>c</b>) Result of ground resistance map with wheel loader (UKF with 1 IMU and 1 GPS); (<b>d</b>) Result of ground resistance map with wheel loader (UKF with 1 IMU and 2 GPS); (<b>e</b>) Predefined two areas of road grade plotted by OpenCV; (<b>f</b>) Result of road grade map with wheel loader (EKF with 1 IMU and 1 GPS); (<b>g</b>) Result of road grade map with wheel loader (UKF with 1 IMU and 1 GPS); (<b>h</b>) Result of road grade map with wheel loader (UKF with 1 IMU and 2 GPS).</p> Full article ">Figure 9 Cont.
<p>The ground truth and estimated maps. In the ground resistance map and road grade map plotted by EKF with 1 IMU and 1 GPS, the spikes caused by infrequent GPS are quite obvious. With an additional GPS sensor fused in Kalman filter, the spikes improve a lot. (<b>a</b>) Predefined five areas of resistance plotted by OpenCV; (<b>b</b>) Result of ground resistance map with wheel loader (EKF with 1 IMU and 1 GPS); (<b>c</b>) Result of ground resistance map with wheel loader (UKF with 1 IMU and 1 GPS); (<b>d</b>) Result of ground resistance map with wheel loader (UKF with 1 IMU and 2 GPS); (<b>e</b>) Predefined two areas of road grade plotted by OpenCV; (<b>f</b>) Result of road grade map with wheel loader (EKF with 1 IMU and 1 GPS); (<b>g</b>) Result of road grade map with wheel loader (UKF with 1 IMU and 1 GPS); (<b>h</b>) Result of road grade map with wheel loader (UKF with 1 IMU and 2 GPS).</p> Full article ">Figure 10
<p>Difference between ground truth and the estimated map. Here we compare the predefined areas and the plotted path, where the white pixels are the wrong plotted grids. (<b>a</b>) Result of EKF with 1 IMU and 1 GPS, (<b>b</b>) Result of UKF with 1 IMU and 1 GPS, (<b>c</b>) Result of UKF with 1 IMU and 2 GPS are the Friction map results, and (<b>d</b>) Result of EKF with 1 IMU and 1 GPS, (<b>e</b>) Result of UKF with 1 IMU and 1 GPS, (<b>f</b>) Result of UKF with 1 IMU and 2 GPS are the road grade map results. UKF shows more accurate positioning capabilities than EKF, and with two GPS fused in the Kalman filter, the wrong located grid is less than with just one GPS signal.</p> Full article ">Figure 10 Cont.
<p>Difference between ground truth and the estimated map. Here we compare the predefined areas and the plotted path, where the white pixels are the wrong plotted grids. (<b>a</b>) Result of EKF with 1 IMU and 1 GPS, (<b>b</b>) Result of UKF with 1 IMU and 1 GPS, (<b>c</b>) Result of UKF with 1 IMU and 2 GPS are the Friction map results, and (<b>d</b>) Result of EKF with 1 IMU and 1 GPS, (<b>e</b>) Result of UKF with 1 IMU and 1 GPS, (<b>f</b>) Result of UKF with 1 IMU and 2 GPS are the road grade map results. UKF shows more accurate positioning capabilities than EKF, and with two GPS fused in the Kalman filter, the wrong located grid is less than with just one GPS signal.</p> Full article ">

15 pages, 4310 KiB  

Open AccessArticle

3D Object Detection Based on Attention and Multi-Scale Feature Fusion

by Minghui Liu, Jinming Ma, Qiuping Zheng, Yuchen Liu and Gang Shi

Sensors 2022, 22(10), 3935; https://doi.org/10.3390/s22103935 - 23 May 2022

Cited by 17 | Viewed by 3721

Abstract

Three-dimensional object detection in the point cloud can provide more accurate object data for autonomous driving. In this paper, we propose a method named MA-MFFC that uses an attention mechanism and a multi-scale feature fusion network with ConvNeXt module to improve the accuracy [...] Read more.

Three-dimensional object detection in the point cloud can provide more accurate object data for autonomous driving. In this paper, we propose a method named MA-MFFC that uses an attention mechanism and a multi-scale feature fusion network with ConvNeXt module to improve the accuracy of object detection. The multi-attention (MA) module contains point-channel attention and voxel attention, which are used in voxelization and 3D backbone. By considering the point-wise and channel-wise, the attention mechanism enhances the information of key points in voxels, suppresses background point clouds in voxelization, and improves the robustness of the network. The voxel attention module is used in the 3D backbone to obtain more robust and discriminative voxel features. The MFFC module contains the multi-scale feature fusion network and the ConvNeXt module; the multi-scale feature fusion network can extract rich feature information and improve the detection accuracy, and the convolutional layer is replaced with the ConvNeXt module to enhance the feature extraction capability of the network. The experimental results show that the average accuracy is 64.60% for pedestrians and 80.92% for cyclists on the KITTI dataset, which is 1.33% and 2.1% higher, respectively, compared with the baseline network, enabling more accurate detection and localization of more difficult objects. Full article

(This article belongs to the Topic Information Sensing Technology for Intelligent/Driverless Vehicle)

►▼ Show Figures

Figure 1

Figure 1
<p>The structure of Voxel R-CNN network with multi-attention module and MFF-ConvNeXt module.</p> Full article ">Figure 2
<p>The structure of the point-channel attention module.</p> Full article ">Figure 3
<p>The structure of voxel attention module. The reshape operation permutes the dimension of the tensor from <math display="inline"><semantics> <mrow> <mi>W</mi> <mo>×</mo> <mi>H</mi> <mo>×</mo> <mi>D</mi> <mo>×</mo> <mi>C</mi> </mrow> </semantics></math> to <math display="inline"><semantics> <mrow> <mi>K</mi> <mo>×</mo> <mn>1</mn> <mo>×</mo> <mi>C</mi> </mrow> </semantics></math>.</p> Full article ">Figure 4
<p>The structure of the ConvNeXt block.</p> Full article ">Figure 5
<p>The structure of the MFF-ConvNeXt module.</p> Full article ">Figure 6
<p>Results of 3D detection on the KITTI validation set. The method proposed in this paper can accurately detect the object in the point cloud. The red box represents the ground truth box of the object, the green box represents the detection result of the car, and the blue and yellow represent the detection result of the pedestrian and the cyclist, respectively.</p> Full article ">Figure 7
<p>Visualization of 3D object detection result produced by Voxel R-CNN and the method proposed by this paper. The red box represents the ground truth box of the object, the green box represents the detection result of the car, and the blue and yellow represent the detection result of the pedestrian and the cyclist. (<b>a</b>) Detection results in obscured scenes. (<b>b</b>) Detection results in a simple traffic scenario.</p> Full article ">

13 pages, 1591 KiB  

Open AccessArticle

Research on Dual-Frequency Electromagnetic False Alarm Interference Effect of a Typical Radar

by Xue Du, Guanghui Wei, Kai Zhao, Hongze Zhao and Xuxu Lyu

Sensors 2022, 22(9), 3574; https://doi.org/10.3390/s22093574 - 7 May 2022

Cited by 4 | Viewed by 1857

Abstract

In order to master the position variation rule of radar false alarm signal under continuous wave (CW) electromagnetic interference and reveal the mechanism of CW on radar, taking a certain type of stepping frequency radar as the research object, theoretical analysis of the [...] Read more.

In order to master the position variation rule of radar false alarm signal under continuous wave (CW) electromagnetic interference and reveal the mechanism of CW on radar, taking a certain type of stepping frequency radar as the research object, theoretical analysis of the imaging mechanism of radar CW electromagnetic interference false alarm signals from the perspective of time-frequency decoupling and receiver signal processing. Secondly, electromagnetic interference injection method is used to test the single-frequency and dual-frequency electromagnetic interference effect of the tested equipment. The results show that under the single frequency CW electromagnetic interference, the sensitive bandwidth of false alarm signal is about ±75 MHz, and the position of false alarm signal irregularity changes. Under the in-band dual-frequency CW electromagnetic interference, the position of non-intermodulation false alarm signal is similar to that of single frequency. However, the distance difference of two non-intermodulation false alarm signals is regular. In addition, the positions of the second-order intermodulation false alarm signals of the tested radar are also regular, and its position changes with the change of the second-order intermodulation frequency difference. Full article

(This article belongs to the Topic Information Sensing Technology for Intelligent/Driverless Vehicle)

►▼ Show Figures

Figure 1

Figure 1
<p>Block diagram of field configuration.</p> Full article ">Figure 2
<p>One-dimensional range image of single frequency electromagnetic interference false alarm signal.</p> Full article ">Figure 3
<p>Results of false alarm sensitive threshold of test equipment.</p> Full article ">Figure 4
<p>Position of false alarm signal under Δ<span class="html-italic">f_j</span>₁ = 0 MHz and Δ<span class="html-italic">f_j</span>₂ = 40 MHz electromagnetic inter-ference.</p> Full article ">Figure 5
<p>One-dimensional range profile of second-order intermodulation false alarm signal.</p> Full article ">

26 pages, 8567 KiB  

Open AccessArticle

A SLAM System with Direct Velocity Estimation for Mechanical and Solid-State LiDARs

by Lu Jie, Zhi Jin, Jinping Wang, Letian Zhang and Xiaojun Tan

Remote Sens. 2022, 14(7), 1741; https://doi.org/10.3390/rs14071741 - 4 Apr 2022

Cited by 7 | Viewed by 3932

Abstract

Simultaneous localization and mapping (SLAM) is essential for intelligent robots operating in unknown environments. However, existing algorithms are typically developed for specific types of solid-state LiDARs, leading to weak feature representation abilities for new sensors. Moreover, LiDAR-based SLAM methods are limited by distortions [...] Read more.

Simultaneous localization and mapping (SLAM) is essential for intelligent robots operating in unknown environments. However, existing algorithms are typically developed for specific types of solid-state LiDARs, leading to weak feature representation abilities for new sensors. Moreover, LiDAR-based SLAM methods are limited by distortions caused by LiDAR ego motion. To address the above issues, this paper presents a versatile and velocity-aware LiDAR-based odometry and mapping (VLOM) system. A spherical projection-based feature extraction module is utilized to process the raw point cloud generated by various LiDARs, hence avoiding the time-consuming adaptation of various irregular scan patterns. The extracted features are grouped into higher-level clusters to filter out smaller objects and reduce false matching during feature association. Furthermore, bundle adjustment is adopted to jointly estimate the poses and velocities for multiple scans, effectively improving the velocity estimation accuracy and compensating for point cloud distortions. Experiments on publicly available datasets demonstrate the superiority of VLOM over other state-of-the-art LiDAR-based SLAM systems in terms of accuracy and robustness. Additionally, the satisfactory performance of VLOM on RS-LiDAR-M1, a newly released solid-state LiDAR, shows its applicability to a wide range of LiDARs. Full article

(This article belongs to the Topic Information Sensing Technology for Intelligent/Driverless Vehicle)

►▼ Show Figures

Figure 1

Figure 1
<p>In feature extraction, the scan line structure is used to determine the neighborhood relationship between points. In the above figure, different scan lines are represented by different colors. Different scan lines can be parallel to one another (<b>a</b>,<b>b</b>), stacked vertically for irregular motion (<b>c</b>), or responsible for different areas (<b>d</b>). Due to the variety of structures, it is difficult to uniformly identify neighbors across scan lines. For example, a point on one scan line in Velodyne HDL-32E may be a neighbor of a point with similar timestamps on another scan line; however, this is not the case in RS-LiDAR-M1.</p> Full article ">Figure 2
<p>Overview of our SLAM system, which has three main parts: a versatile feature extraction module, a LiDAR odometry module, and a LiDAR mapping module.</p> Full article ">Figure 3
<p>Our edge feature selection scheme. An intersection edge is a planar point with a smaller depth than its neighboring points and a larger angle, e.g., <math display="inline"><semantics> <mrow> <mn>90</mn> <mo>°</mo> </mrow> </semantics></math>, between the corresponding normal vectors. A jump edge is a plane point with a much smaller depth than its neighboring points. A thin edge is a point with two neighbors in the same row or column that have much larger depths. The depth is infinite if the yellow plane with the dashed boundary does not exist.</p> Full article ">Figure 4
<p>A map in 3D space. The two spheres on the right show the details of the planar and edge features. The different colored points in the spheres correspond to the LiDAR scans of the same color.</p> Full article ">Figure 5
<p>Illustration of our graph for odometry optimization. The first pose serves as a prior and is fixed during the optimization process. The planar and edge features impose constraints on all scans that observe them, while the constant motion model imposes constraints on neighboring scans.</p> Full article ">Figure 6
<p>Comparison of the feature extraction results with Lego-LOAM and VLOM. (<b>a</b>) Features extracted with Lego-LOAM and (<b>b</b>) features extracted with VLOM. The green dots indicate planar feature points, while the red dots indicate the edge feature points. We marked some regions at the corresponding positions to compare the details of the two results.</p> Full article ">Figure 7
<p>Segmentation results of Lego-LOAM (<b>a</b>) and VLOM (<b>b</b>). The various colors represent different planes. The rectangle in the image indicates some differences between Lego-LOAM and VLOM. The continuous color in Lego-LOAM indicates that each plane is divided into several parts.</p> Full article ">Figure 8
<p>(<b>a</b>) compares the estimated trajectories with the ground-truth trajectories in the HK-Data20190117 dataset. Note that all trajectories were aligned to the ground truth according to the EVO library. (<b>b</b>) shows a point cloud acquired at the position where A-LOAM begins to fail, which is marked by the circle in (<b>a</b>).</p> Full article ">Figure 9
<p>There are clearly large errors in the origins of BALM (<b>a</b>) and A-LOAM (<b>b</b>). By replacing the feature extraction module with our scheme, the new algorithms, BALM-F (<b>a</b>) and A-LOAM-F (<b>b</b>), achieve impressive performance improvements.</p> Full article ">Figure 10
<p>Comparison of the linear velocity errors on <span class="html-italic">HK-Data20190117</span>. VLOM has a very low error throughout the sequence, especially during the initial phase shown in the magnified area. Lego-LOAM is more accurate than the other methods; however, it has a poor performance in the areas marked by the gray dotted line.</p> Full article ">Figure 11
<p>Mapping results of our approach on <span class="html-italic">Schloss-2</span>, which are rendered based on the intensity value: (<b>a</b>) top view, (<b>b</b>) side view. Our method produces a map with fine details, especially in the magnified region of (<b>b</b>).</p> Full article ">Figure 12
<p>Experimental platform. (<b>a</b>) Front view. (<b>b</b>) Side view. Ouster OS1-64 and RS-LiDAR-M1 were placed on the roof and front of the vehicle, respectively.</p> Full article ">Figure 13
<p>Trajectory of the dataset acquired with RS-LiDAR-M1. (<b>a</b>) Ground truth aligned to Google Earth from a top view. (<b>b</b>) Comparison of the VLOM trajectory and the ground truth. The results are presented in meters.</p> Full article ">Figure 14
<p>Changes in the position and orientation contrasted with the ground truth on the <span class="html-italic">RS</span> dataset.</p> Full article ">Figure 15
<p>Comparison of velocity errors on the <span class="html-italic">OS</span> dataset, including the linear velocity error (<b>top</b>) and angular velocity error (<b>bottom</b>).</p> Full article ">

20 pages, 895 KiB  

Open AccessArticle

Multi-Target Localization of MIMO Radar with Widely Separated Antennas on Moving Platforms Based on Expectation Maximization Algorithm

by Jiaxin Lu, Feifeng Liu, Jingyi Sun, Yingjie Miao and Quanhua Liu

Remote Sens. 2022, 14(7), 1670; https://doi.org/10.3390/rs14071670 - 30 Mar 2022

Cited by 6 | Viewed by 2108

Abstract

This paper focuses on multi-target parameter estimation of multiple-input multiple-output (MIMO) radar with widely separated antennas on moving platforms. Aiming at the superimposed signals caused by multi-targets, the well-known expectation maximization (EM) is used in this paper. Target’s radar cross-section (RCS) spatial variations, [...] Read more.

This paper focuses on multi-target parameter estimation of multiple-input multiple-output (MIMO) radar with widely separated antennas on moving platforms. Aiming at the superimposed signals caused by multi-targets, the well-known expectation maximization (EM) is used in this paper. Target’s radar cross-section (RCS) spatial variations, different path losses and spatially-non-white noise appear because of the widely separated antennas. These variables are collectively referred to as signal-to-noise ratio (SNR) fluctuations. To estimate the echo delay/Doppler shift and SNR, the Q function of EM algorithm is extended. In addition, to reduce the computational complexity of EM algorithm, the gradient descent is used in M-step of EM algorithm. The modified EM algorithm is called generalized adaptive EM (GAEM) algorithm. Then, a weighted iterative least squares (WILS) algorithm is used to jointly estimate the target positions and velocities based on the results of GAEM algorithm. This paper also derives the Cramér-Rao bound (CRB) in such a non-ideal environment. Finally, extensive numerical simulations are carried out to validate the effectiveness of the proposed algorithm. Full article

(This article belongs to the Topic Information Sensing Technology for Intelligent/Driverless Vehicle)

►▼ Show Figures

Figure 1

Figure 1
<p>System diagram of multiple-input multiple-output (MIMO) radar with widely separated antennas on moving platforms.</p> Full article ">Figure 2
<p>Topology diagram of MIMO radar and targets, which shows two practical problems concerned in this paper: (1) target echo overlap, (2) different signal-to-noise ratios (SNRs) in different propagation paths.</p> Full article ">Figure 3
<p>The generalized adaptive expectation maximization (GAEM) algorithm, which is accelerated by squared iterative methods (SQUAREM) algorithm.</p> Full article ">Figure 4
<p>Objective function of echo delay and Doppler shift.</p> Full article ">Figure 5
<p>Simulation of the superimposed signal. (<b>a</b>) The distance between two targets is 60 m. (<b>b</b>) The distance between two targets is 30 m.</p> Full article ">Figure 6
<p>Estimation results for proposed GAEM and gradient descent method based on traditional maximum likelihood (ML) [<a href="#B34-remotesensing-14-01670" class="html-bibr">34</a>] in the presence of target echo superimposed. (<b>a</b>) Root mean square error (RMSE) for the echo delay estimation, (<b>b</b>) RMSE for the echo Doppler estimation.</p> Full article ">Figure 7
<p>Computational complexity versus grid resolution for the GAEM and traditional EM.</p> Full article ">Figure 8
<p>Deployment of radar nodes and targets.</p> Full article ">Figure 9
<p>Estimation results for different methods in the presence of different SNRs in different propagation paths (<b>a</b>) RMSE for the DOA estimation, (<b>b</b>) RMSE for the target range estimation, (<b>c</b>) RMSE for the target tangential velocity estimation, (<b>d</b>) RMSE for the target radial velocity estimation.</p> Full article ">Figure 10
<p>Estimation results for different methods in the presence of different SNRs in different propagation paths (<b>a</b>) RMSE for the radar position deviations estimation, (<b>b</b>) RMSE for the radar velocity deviations estimation.</p> Full article ">

18 pages, 14969 KiB  

Open AccessArticle

A Hierarchical Path Planning Approach with Multi-SARSA Based on Topological Map

by Shiguang Wen, Yufan Jiang, Ben Cui, Ke Gao and Fei Wang

Sensors 2022, 22(6), 2367; https://doi.org/10.3390/s22062367 - 18 Mar 2022

Cited by 16 | Viewed by 2940

Abstract

In this paper, a novel path planning algorithm with Reinforcement Learning is proposed based on the topological map. The proposed algorithm has a two-level structure. At the first level, the proposed method generates the topological area using the region dynamic growth algorithm based [...] Read more.

In this paper, a novel path planning algorithm with Reinforcement Learning is proposed based on the topological map. The proposed algorithm has a two-level structure. At the first level, the proposed method generates the topological area using the region dynamic growth algorithm based on the grid map. In the next level, the Multi-SARSA method divided into two layers is applied to find a near-optimal global planning path, in which the artificial potential field method, first of all, is used to initialize the first Q table for faster learning speed, and then the second Q table is initialized with the connected domain obtained by topological map, which provides the prior information. A combination of the two algorithms makes the algorithm easier to converge. Simulation experiments for path planning have been executed. The results indicate that the method proposed in this paper can find the optimal path with a shorter path length, which demonstrates the effectiveness of the presented method. Full article

(This article belongs to the Topic Information Sensing Technology for Intelligent/Driverless Vehicle)

►▼ Show Figures

Figure 1

Figure 1
<p>The schematic diagram of path planning mathematical model based on topological map. In the figure, <math display="inline"><semantics> <mfenced separators="" open="(" close=")"> <mrow> <msub> <mi>x</mi> <mn>0</mn> </msub> <mo>,</mo> <msub> <mi>y</mi> <mn>0</mn> </msub> </mrow> </mfenced> </semantics></math> is the starting point, <math display="inline"><semantics> <mfenced separators="" open="(" close=")"> <mrow> <msub> <mi>x</mi> <mi>t</mi> </msub> <mo>,</mo> <msub> <mi>y</mi> <mi>t</mi> </msub> </mrow> </mfenced> </semantics></math> is the the end point and the green points 1–18 are the topological nodes representing the topological areas.</p> Full article ">Figure 2
<p>The framework of the proposed method.</p> Full article ">Figure 3
<p>The steps of topological map generation in a simulated environment.</p> Full article ">Figure 4
<p>The framework of Multi-SARSA algorithm structurally improved, in which the left column is the first layer and the right column is the second layer, and the a priori information is provided by the topological map.</p> Full article ">Figure 5
<p>The path planning results of artificial potential field method on MATLAB: (<b>a</b>) Set the green dot in the upper right corner as the starting point and the yellow dot in the lower left corner as the end point, while the three-dimensional graphs of the attractive field, repulsive force field, and the total force field on the right side. Compared with (<b>a</b>,<b>b</b>) moved the starting point, and (<b>c</b>) exchanged the starting point and the end point. The result is shown in the figure.</p> Full article ">Figure 6
<p>(<b>a</b>) The four directions that the robot can move corresponding to four actions. (<b>b</b>) The way to convert angles to actions, which 360 degrees are divided into four sections, that is, <math display="inline"><semantics> <mrow> <mfenced separators="" open="[" close=""> <mrow> <mo>−</mo> <mi>π</mi> <mspace width="-0.166667em"/> <mrow> <mfenced open="/" close=""> <mphantom> <mpadded width="0pt"> <mi>π</mi> <mn>4</mn> </mpadded> </mphantom> </mfenced> <mspace width="0.0pt"/> </mrow> <mspace width="-0.166667em"/> <mn>4</mn> </mrow> </mfenced> <mo>,</mo> <mi>π</mi> <mspace width="-0.166667em"/> <mrow> <mfenced open="/" close=""> <mphantom> <mpadded width="0pt"> <mi>π</mi> <mn>4</mn> </mpadded> </mphantom> </mfenced> <mspace width="0.0pt"/> </mrow> <mspace width="-0.166667em"/> </mrow> </semantics></math>4], [<math display="inline"><semantics> <mi>π</mi> </semantics></math>/4, <math display="inline"><semantics> <mrow> <mn>3</mn> <mi>π</mi> </mrow> </semantics></math>/4], [<math display="inline"><semantics> <mrow> <mn>3</mn> <mi>π</mi> </mrow> </semantics></math>/4, −<math display="inline"><semantics> <mrow> <mn>3</mn> <mi>π</mi> </mrow> </semantics></math>/4], [−<math display="inline"><semantics> <mi>π</mi> </semantics></math>/4, −<math display="inline"><semantics> <mrow> <mn>3</mn> <mi>π</mi> </mrow> </semantics></math>/4].</p> Full article ">Figure 7
<p>The successfully planned path in MATLAB corresponds to <a href="#sensors-22-02367-t001" class="html-table">Table 1</a>. The three blue circles in the figure are obstacles, and the red dots are the successfully planned paths.</p> Full article ">Figure 8
<p>The (<b>left picture</b>) is an angle diagram inspired by DWA, and <math display="inline"><semantics> <mrow> <msub> <mi>θ</mi> <mn>1</mn> </msub> <mo>−</mo> <msub> <mi>θ</mi> <mn>0</mn> </msub> </mrow> </semantics></math> in the picture is the angle difference needed; the (<b>middle picture</b>) is a comparison diagram of Euclidean distance and Manhattan distance; and the (<b>right picture</b>) is a distance diagram, in which <math display="inline"><semantics> <mrow> <mi>d</mi> <mi>i</mi> <mi>a</mi> <mo form="prefix">tan</mo> <mi>c</mi> <mi>e</mi> <mspace width="-2.0pt"/> <mo>_</mo> <mspace width="-2.0pt"/> <mn>1</mn> <mspace width="1.0pt"/> <mspace width="1.0pt"/> <mo>−</mo> <mspace width="1.0pt"/> <mspace width="1.0pt"/> <mspace width="1.0pt"/> <mi>d</mi> <mi>i</mi> <mi>s</mi> <mo form="prefix">tan</mo> <mi>c</mi> <mi>e</mi> <mspace width="-2.0pt"/> <mo>_</mo> <mspace width="-2.0pt"/> <mn>0</mn> </mrow> </semantics></math> is <math display="inline"><semantics> <mrow> <mi>d</mi> <mi>i</mi> <mi>a</mi> <mo form="prefix">tan</mo> <mi>c</mi> <mi>e</mi> <mo>(</mo> <mi>v</mi> <mo>,</mo> <mi>w</mi> <mo>)</mo> </mrow> </semantics></math> in the function.</p> Full article ">Figure 9
<p>Experimental results of the artificial potential field method under similar scenes and the same starting point and end point settings (failed).</p> Full article ">Figure 10
<p>The experimental results of the search algorithm in a similar environment: (<b>a</b>) the experimental environment; (<b>b</b>) the planning result of the A* algorithm based on Euclidean distance (success); (<b>c</b>) the planning of the IDA* algorithm also based on the Euclidean distance result (failure).</p> Full article ">Figure 11
<p>The topological map construction process in a similar environment (basically the same as the process in <a href="#sensors-22-02367-f003" class="html-fig">Figure 3</a>). (<b>a</b>) is the simulated environment in Gazebo, (<b>b</b>) is the trajectory of robot, (<b>c</b>) is the nodes generated on the trajectory, (<b>d</b>) is the adjacency, (<b>e</b>) is the region dynamic growth algorithm and (<b>f</b>) is the Generated topological area.</p> Full article ">Figure 12
<p>(<b>a</b>) The simulation environment generated according to <a href="#sensors-22-02367-f011" class="html-fig">Figure 11</a> (same as <a href="#sensors-22-02367-f010" class="html-fig">Figure 10</a>a). (<b>b</b>) Q-table updated according to the topological relationship in (<b>a</b>).</p> Full article ">Figure 13
<p>In the same environment, (<b>a</b>–<b>c</b>) are the planning results of the SARSA algorithm, Q-Learning, and the algorithm proposed in this paper, respectively. The colored part is a topological area, and one color corresponds to a topological area. The white dot is the selected representative topological node, the starting point is the upper left corner, and the end point is set in the lower right corner. (<b>a</b>,<b>d</b>) are the paths generated by sarsa algorithm, (<b>b</b>,<b>e</b>) are the paths generated by q-learning algorithm, and (<b>c</b>,<b>f</b>) are the paths generated by the multi-sarsa algorithm.</p> Full article ">Figure 14
<p>The step diagram of the Multi-SARSA algorithm.</p> Full article ">Figure 15
<p>A path planned according to people’s preferences (1-2-4-10-16-17).</p> Full article ">

18 pages, 4149 KiB  

Open AccessArticle

Spatial Attention Frustum: A 3D Object Detection Method Focusing on Occluded Objects

by Xinglei He, Xiaohan Zhang, Yichun Wang, Hongzeng Ji, Xiuhui Duan and Fen Guo

Sensors 2022, 22(6), 2366; https://doi.org/10.3390/s22062366 - 18 Mar 2022

Viewed by 2760

Abstract

Achieving the accurate perception of occluded objects for autonomous vehicles is a challenging problem. Human vision can always quickly locate important object regions in complex external scenes, while other regions are only roughly analysed or ignored, defined as the visual attention mechanism. However, [...] Read more.

Achieving the accurate perception of occluded objects for autonomous vehicles is a challenging problem. Human vision can always quickly locate important object regions in complex external scenes, while other regions are only roughly analysed or ignored, defined as the visual attention mechanism. However, the perception system of autonomous vehicles cannot know which part of the point cloud is in the region of interest. Therefore, it is meaningful to explore how to use the visual attention mechanism in the perception system of autonomous driving. In this paper, we propose the model of the spatial attention frustum to solve object occlusion in 3D object detection. The spatial attention frustum can suppress unimportant features and allocate limited neural computing resources to critical parts of the scene, thereby providing greater relevance and easier processing for higher-level perceptual reasoning tasks. To ensure that our method maintains good reasoning ability when faced with occluded objects with only a partial structure, we propose a local feature aggregation module to capture more complex local features of the point cloud. Finally, we discuss the projection constraint relationship between the 3D bounding box and the 2D bounding box and propose a joint anchor box projection loss function, which will help to improve the overall performance of our method. The results of the KITTI dataset show that our proposed method can effectively improve the detection accuracy of occluded objects. Our method achieves 89.46%, 79.91% and 75.53% detection accuracy in the easy, moderate, and hard difficulty levels of the car category, and achieves a 6.97% performance improvement especially in the hard category with a high degree of occlusion. Our one-stage method does not need to rely on another refining stage, comparable to the accuracy of the two-stage method. Full article

(This article belongs to the Topic Information Sensing Technology for Intelligent/Driverless Vehicle)

►▼ Show Figures

Figure 1

Figure 1
<p>Common occlusion scene in autonomous driving.</p> Full article ">Figure 2
<p>The frustum with the same length at each scale means that the unimportant point cloud and the attention point cloud cannot be effectively distinguished. The ‘+’ is concatenate operation.</p> Full article ">Figure 3
<p>The feature vector of the unimportant object will seriously affect the expression of the feature vector of the object of interest in feature map. The ‘+’ is concatenate operation.</p> Full article ">Figure 4
<p>The frustum with spatial attention can improve the feature expression of the focused objects in the feature map. The ‘+’ is concatenate operation.</p> Full article ">Figure 5
<p>Projection relationship between ground truth and image.</p> Full article ">Figure 6
<p>The LFA module.</p> Full article ">Figure 7
<p>The effect of different modules on performance improvement.</p> Full article ">Figure 8
<p>The precision-recall curves for car 3D detection at all levels of difficulty.</p> Full article ">Figure 9
<p>Qualitative results on the KITTI.</p> Full article ">

18 pages, 14069 KiB  

Open AccessArticle

Development of a GPU-Accelerated NDT Localization Algorithm for GNSS-Denied Urban Areas

by Keon Woo Jang, Woo Jae Jeong and Yeonsik Kang

Sensors 2022, 22(5), 1913; https://doi.org/10.3390/s22051913 - 1 Mar 2022

Cited by 6 | Viewed by 3485

Abstract

There are numerous global navigation satellite system-denied regions in urban areas, where the localization of autonomous driving remains a challenge. To address this problem, a high-resolution light detection and ranging (LiDAR) sensor was recently developed. Various methods have been proposed to improve the [...] Read more.

There are numerous global navigation satellite system-denied regions in urban areas, where the localization of autonomous driving remains a challenge. To address this problem, a high-resolution light detection and ranging (LiDAR) sensor was recently developed. Various methods have been proposed to improve the accuracy of localization using precise distance measurements derived from LiDAR sensors. This study proposes an algorithm to accelerate the computational speed of LiDAR localization while maintaining the original accuracy of lightweight map-matching algorithms. To this end, first, a point cloud map was transformed into a normal distribution (ND) map. During this process, vector-based normal distribution transform, suitable for graphics processing unit (GPU) parallel processing, was used. In this study, we introduce an algorithm that enabled GPU parallel processing of an existing ND map-matching process. The performance of the proposed algorithm was verified using an open dataset and simulations. To verify the practical performance of the proposed algorithm, the real-time serial and parallel processing performances of the localization were compared using high-performance and embedded computers, respectively. The distance root-mean-square error and computational time of the proposed algorithm were compared. The algorithm increased the computational speed of the embedded computer almost 100-fold while maintaining high localization precision. Full article

(This article belongs to the Topic Information Sensing Technology for Intelligent/Driverless Vehicle)

►▼ Show Figures

Figure 1

Figure 1
<p>Flowchart of GPU-accelerated NDT localization algorithm.</p> Full article ">Figure 2
<p>Bird’s-eye views of the urban area implemented in the CarMaker simulation. (<b>a</b>) CarMaker simulation viewer. (<b>b</b>) 3D point cloud map.</p> Full article ">Figure 3
<p>Comparison of road lane information. (<b>a</b>) 3D point cloud map. (<b>b</b>) High-definition road map.</p> Full article ">Figure 4
<p>Vector-based NDT process of representing urban buildings. (<b>a</b>–<b>c</b>) Representative building layers. (<b>d</b>) ND map of the building.</p> Full article ">Figure 5
<p>(<b>a</b>) Elements used for the generation of the vector-based ND map. (<b>b</b>) Part of vector-based ND map and point cloud map.</p> Full article ">Figure 6
<p>Flowchart of the map-matching process.</p> Full article ">Figure 7
<p>Operational flow of the parallel cost calculation.</p> Full article ">Figure 8
<p>(<b>a</b>) Depiction of using GPU thread and block. (<b>b</b>) Operational flow of the parallel map searching.</p> Full article ">Figure 9
<p>Driving routes (i.e., red arrow) of the selected nuScenes scenes. (<b>a</b>) scene-0061, (<b>b</b>) scene-0103, and (<b>c</b>) scene-1094.</p> Full article ">Figure 10
<p>Comparison of the urban area implementation in CarMaker. (<b>a</b>) Real urban view. (<b>b</b>) CarMaker simulation viewer.</p> Full article ">Figure 11
<p>Position of the sensors attached to the simulation vehicle.</p> Full article ">Figure 12
<p>Verification route (i.e., red arrow) of CarMaker.</p> Full article ">Figure 13
<p>Comparison between the localization results and the reference (<b>top</b>) and LiDAR map-matching results (<b>bottom</b>) in nuScenes. (<b>a</b>) scene-0061, (<b>b</b>) scene-0103, and (<b>c</b>) scene-1094.</p> Full article ">Figure 14
<p>Localization result from the CarMaker simulation.</p> Full article ">Figure 15
<p>The average computation time for each function in the CarMaker simulation. (<b>a</b>) Transform point cloud. (<b>b</b>) Map searching. (<b>c</b>) Cost calculation.</p> Full article ">Figure 16
<p>Computation time for each function in the CarMaker simulation. (<b>a</b>) Transform point cloud. (<b>b</b>) Map searching. (<b>c</b>) Cost calculation.</p> Full article ">

12 pages, 29144 KiB  

Open AccessArticle

Machine Vision-Based Method for Estimating Lateral Slope of Structured Roads

by Yunbing Yan and Haiwei Li

Sensors 2022, 22(5), 1867; https://doi.org/10.3390/s22051867 - 26 Feb 2022

Cited by 3 | Viewed by 2805

Abstract

Most of the studies on vehicle control and stability are based on cases of known-road lateral slope, while there are few studies on road lateral-slope estimation. In order to provide reliable information on slope parameters for subsequent studies, this paper provides a method [...] Read more.

Most of the studies on vehicle control and stability are based on cases of known-road lateral slope, while there are few studies on road lateral-slope estimation. In order to provide reliable information on slope parameters for subsequent studies, this paper provides a method of structured-road lateral-slope estimation based on machine vision. The relationship between the road lateral slope and the tangent slope of the lane line can be found out according to the image-perspective principle; then, the coordinates of the pre-scan point are obtained by the lane line, and the tangent slope of the lane line is used to obtain a more accurate estimation of the road lateral slope. In the implementation process, the lane-line feature information in front of the vehicle is obtained according to machine vision, the lane-line function is fitted according to an SCNN (Spatial CNN) algorithm, then the lateral slope is calculated by using the estimation formula mentioned above. Finally, the road model and vehicle model are established by Prescan software for off-line simulation. The simulation results verify the effectiveness and accuracy of the method. Full article

(This article belongs to the Topic Information Sensing Technology for Intelligent/Driverless Vehicle)

►▼ Show Figures

Figure 1

Figure 1
<p>Schematic diagram of the geometric relationship between coordinate systems.</p> Full article ">Figure 2
<p>Schematic diagram of wedge-shaped pavement.</p> Full article ">Figure 3
<p>Overhead diagram of curved road section.</p> Full article ">Figure 4
<p>Effect of SCNN lane line detection algorithm.</p> Full article ">Figure 5
<p>Diagram of lane line projection.</p> Full article ">Figure 6
<p>Schematic diagram of the experimental part of PRESCAN <math display="inline"><semantics> <mrow> <msub> <mi>d</mi> <mi>l</mi> </msub> </mrow> </semantics></math> and <math display="inline"><semantics> <mrow> <msub> <mi>d</mi> <mi>r</mi> </msub> </mrow> </semantics></math> estimation.</p> Full article ">Figure 7
<p>Plot of experimental results of <math display="inline"><semantics> <mrow> <msub> <mi>d</mi> <mi>l</mi> </msub> <mo> </mo> </mrow> </semantics></math> estimation.</p> Full article ">Figure 8
<p>Schematic diagram of the experimental part of Prescan lane-line tangent-slope estimation.</p> Full article ">Figure 9
<p>Graph of lane-line tangent-slope estimation results.</p> Full article ">Figure 10
<p>Schematic diagram of the experimental part of Prescan slope estimation.</p> Full article ">Figure 11
<p>Graph of the results of estimating the lateral-slope angle of a straight road.</p> Full article ">Figure 12
<p>Estimation results of lateral-slope angle of curved road.</p> Full article ">Figure 13
<p>Error effect diagram of <math display="inline"><semantics> <mrow> <msub> <mi>K</mi> <mi>l</mi> </msub> </mrow> </semantics></math>.</p> Full article ">

11 pages, 445 KiB  

Open AccessTechnical Note

Target Localization Based on High Resolution Mode of MIMO Radar with Widely Separated Antennas

by Jiaxin Lu, Feifeng Liu, Hongjie Liu and Quanhua Liu

Remote Sens. 2022, 14(4), 902; https://doi.org/10.3390/rs14040902 - 14 Feb 2022

Cited by 3 | Viewed by 1962

Abstract

Coherent processing of multiple-input multiple-output (MIMO) radar with widely separated antennas has high resolution capability, but it also brings ambiguity in target localization. In view of the ambiguity problem, different from other signal processing sub-directions such as array configuration optimization or continuity of [...] Read more.

Coherent processing of multiple-input multiple-output (MIMO) radar with widely separated antennas has high resolution capability, but it also brings ambiguity in target localization. In view of the ambiguity problem, different from other signal processing sub-directions such as array configuration optimization or continuity of phase in space/time, this paper analyzes it from the information level, that is, the tracking method is adopted. First, by using the state equation and measurement equation, the echo data of multiple coherent processing intervals (CPI) are collected to improve the target localization accuracy as much as possible. Second, the non-coherent joint probability data association filter (JPDAF) is used to achieve stable tracking of spatial cross targets without ambiguity measurements. Third, based on the tracking results of the non-coherent JPDAF, the ambiguity of coherent measurement is resolved, that is, the coherent JPDAF is realized. By means of non-coherent and coherent alternating JPDAF (NCCAF) algorithms, high accuracy localization of multiple targets is achieved. Finally, numerical simulations are carried out to validate the effectiveness of the proposed NCCAF algorithm. Full article

(This article belongs to the Topic Information Sensing Technology for Intelligent/Driverless Vehicle)

►▼ Show Figures

Graphical abstract

Graphical abstract
Full article ">Figure 1
<p>The scenario of multiple targets in spatial crossing and ambiguous measurements.</p> Full article ">Figure 2
<p>Topology of multi-target tracking.</p> Full article ">Figure 3
<p>The root mean square error (RMSE) results of the tracking algorithm.</p> Full article ">

12 pages, 32750 KiB  

Open AccessTechnical Note

Research of Distance-Intensity Imaging Algorithm for Pulsed LiDAR Based on Pulse Width Correction

by Shiyu Yan, Guohui Yang, Qingyan Li, Yue Wang and Chunhui Wang

Remote Sens. 2022, 14(3), 507; https://doi.org/10.3390/rs14030507 - 21 Jan 2022

Cited by 4 | Viewed by 2586

Abstract

Walking error has been problematic for pulsed LiDAR based on a single threshold comparator. Traditionally, walk error must be suppressed by some time discrimination methods with extremely complex electronic circuits and high costs. In this paper, we propose a compact and flexible method [...] Read more.

Walking error has been problematic for pulsed LiDAR based on a single threshold comparator. Traditionally, walk error must be suppressed by some time discrimination methods with extremely complex electronic circuits and high costs. In this paper, we propose a compact and flexible method for reducing walk error and achieving distance-intensity imaging. A single threshold comparator and commercial time digital converter chip are designed to measure the laser pulse’s time of flight and pulse width. In order to obtain first-class measurement accuracy, we designed a specific pulse width correction method based on the Kalman filter to correct the laser recording time, significantly reducing the ranging walk error by echo intensity fluctuation. In addition, the pulse width obtained by our method, which is a recording of the laser intensity, is conducive to target identification. The experiment results verified plane point clouds of various targets obtained by the proposed method with a plane flatness less than 0.34. The novel contribution of the study is to provide a highly integrated and cost-effective solution for the realization of high-precision ranging and multi-dimensional detection by pulsed LiDAR. It is valuable for realizing multi-dimension, outstanding performance, and low-cost LiDAR. Full article

(This article belongs to the Topic Information Sensing Technology for Intelligent/Driverless Vehicle)

►▼ Show Figures

Figure 1

Figure 1
<p>A structure chart of the pulsed LiDAR system.</p> Full article ">Figure 2
<p>Transmitted pulse and echo pulses with various amplitudes.</p> Full article ">Figure 3
<p>Echo pulses with various amplitudes and pulse widths.</p> Full article ">Figure 4
<p>The principle diagram of time measurement.</p> Full article ">Figure 5
<p>The flow chart of the proposed algorithm.</p> Full article ">Figure 6
<p>The experimental pulsed LiDAR. (<b>a</b>) External view; (<b>b</b>) Internal MCU and TDC circuit board.</p> Full article ">Figure 7
<p>The echo signals with various pump power control voltages with a target of 50% diffuse reflection plate at 5.51 m.</p> Full article ">Figure 8
<p>Standard error map of time ranging results with two methods at various diffuse panels of the same distance.</p> Full article ">Figure 9
<p>Testing environments 1. (<b>a</b>) Picture of diffuse reflector; (<b>b</b>) Point cloud of diffuse reflector by LD; (<b>c</b>) Point cloud of diffuse reflector by CFD; (<b>d</b>) Point cloud of diffuse reflector by our method.</p> Full article ">Figure 10
<p>Testing environments 2. (<b>a</b>) Picture of diffuse reflector; (<b>b</b>) Point cloud of diffuse reflector by LD; (<b>c</b>) Point cloud of diffuse reflector by CFD; (<b>d</b>) Point cloud of diffuse reflector by our method.</p> Full article ">

15 pages, 3196 KiB  

Open AccessArticle

Vehicle Destination Prediction Using Bidirectional LSTM with Attention Mechanism

by Pietro Casabianca, Yu Zhang, Miguel Martínez-García and Jiafu Wan

Sensors 2021, 21(24), 8443; https://doi.org/10.3390/s21248443 - 17 Dec 2021

Cited by 10 | Viewed by 3361

Abstract

Satellite navigation has become ubiquitous to plan and track travelling. Having access to a vehicle’s position enables the prediction of its destination. This opens the possibility to various benefits, such as early warnings of potential hazards, route diversions to pass traffic congestion, and [...] Read more.

Satellite navigation has become ubiquitous to plan and track travelling. Having access to a vehicle’s position enables the prediction of its destination. This opens the possibility to various benefits, such as early warnings of potential hazards, route diversions to pass traffic congestion, and optimizing fuel consumption for hybrid vehicles. Thus, reliably predicting destinations can bring benefits to the transportation industry. This paper investigates using deep learning methods for predicting a vehicle’s destination based on its journey history. With this aim, Dense Neural Networks (DNNs), Long Short-Term Memory (LSTM) networks, Bidirectional LSTM (BiLSTM), and networks with and without attention mechanisms are tested. Especially, LSTM and BiLSTM models with attention mechanism are commonly used for natural language processing and text-classification-related applications. On the other hand, this paper demonstrates the viability of these techniques in the automotive and associated industrial domain, aimed at generating industrial impact. The results of using satellite navigation data show that the BiLSTM with an attention mechanism exhibits better prediction performance destination, achieving an average accuracy of 96% against the test set (4% higher than the average accuracy of the standard BiLSTM) and consistently outperforming the other models by maintaining robustness and stability during forecasting. Full article

(This article belongs to the Topic Information Sensing Technology for Intelligent/Driverless Vehicle)

►▼ Show Figures

Figure 1

Figure 1
<p>Car trajectories around the capitol of China, Beijing.</p> Full article ">Figure 2
<p>Number of journeys reaching each specific destination cluster. A destination where one journey was removed prior to model training (yellow) is used as an unseen full journey test.</p> Full article ">Figure 3
<p>Training curve of a BiLSTM with an attention mechanism. The lowest validation loss occurs at epoch 63 and this model is saved.</p> Full article ">Figure 4
<p>Model accuracy evaluations for ten training and testing runs shown by boxplots and × indicating the mean.</p> Full article ">Figure 5
<p>(<b>a</b>) Correct destination prediction versus percentage complete for the unseen journey to Destination 22. (<b>b</b>) BiLSTM with Attention destination predictions visualized along the same route. The red pin is the starting point. Map data © 2021.</p> Full article ">Figure 6
<p>(<b>a</b>) Correct destination prediction certainty versus percentage complete for the seen journey to Destination 5. (<b>b</b>) BiLSTM with Attention destination predictions visualized along the same route. The red pin is the starting point. Map data © 2021.</p> Full article ">Figure 7
<p>(<b>a</b>) Correct destination prediction certainty versus percentage complete for the seen journey to Destination 15. (<b>b</b>) BiLSTM with Attention destination predictions visualized along the same route. The red pin is the starting point. Map data © 2021.</p> Full article ">

16 pages, 4787 KiB  

Open AccessArticle

Position Estimation of Vehicle Based on Magnetic Marker: Time-Division Position Correction

by Yeun Sub Byun and Rag Gyo Jeong

Sensors 2021, 21(24), 8274; https://doi.org/10.3390/s21248274 - 10 Dec 2021

Viewed by 3106

Abstract

During the automatic driving of a vehicle, the vehicle’s positional information is important for vehicle driving control. If fixed-point land markers such as magnetic markers are used, the vehicle’s current position error can be calculated only when a marker is detected while driving, [...] Read more.

During the automatic driving of a vehicle, the vehicle’s positional information is important for vehicle driving control. If fixed-point land markers such as magnetic markers are used, the vehicle’s current position error can be calculated only when a marker is detected while driving, and this error can be used to correct the estimation position. Therefore, correction information is used irregularly and intermittently according to the installation intervals of the magnetic markers and the driving speed. If the detected errors are corrected all at once using the position correction method, discontinuity of the position information can occur. This problem causes instability in the vehicle’s route guidance control because the position error fluctuates as the vehicle’s speed increases. We devised a time-division position correction method that calculates the error using the absolute position of the magnetic marker, which is estimated when the magnetic marker is detected, along with the absolute position information from the magnetic marker database. Instead of correcting the error at once when the position and heading errors are corrected, the correction is performed by dividing the errors multiple times until the next magnetic marker is detected. This prevents sudden discontinuity of the vehicle position information, and the calculated correction amount is used without loss to obtain stable and continuous position information. We conducted driving tests to compare the performances of the proposed algorithm and conventional methods. We compared the continuity of the position information and the mean error and confirmed the superiority of the proposed method in terms of these aspects. Full article

(This article belongs to the Topic Information Sensing Technology for Intelligent/Driverless Vehicle)

►▼ Show Figures

Figure 1

Figure 1
<p>Schematic of the automatic guidance system based on magnetic markers.</p> Full article ">Figure 2
<p>Installation of the magnetic sensing ruler (MSR).</p> Full article ">Figure 3
<p>Magnetic sensing ruler (MSR).</p> Full article ">Figure 4
<p>Magnetic marker and magnetic sensing ruler.</p> Full article ">Figure 5
<p>Magnetic field signal collected by the magnetic sensing ruler.</p> Full article ">Figure 6
<p>Peak of magnetic field signal in the 2D data space.</p> Full article ">Figure 7
<p>Magnet signal peak for each axis.</p> Full article ">Figure 8
<p>Vehicle local Cartesian coordinate system.</p> Full article ">Figure 9
<p>Magnetic marker center position in the sensor coordinate system.</p> Full article ">Figure 10
<p>Kinematic model of the vehicle.</p> Full article ">Figure 11
<p>Measurement model with MSR.</p> Full article ">Figure 12
<p>Magnetic-marker-based position estimation and guidance control test results.</p> Full article ">Figure 13
<p>Absolute position error based on the magnetic marker.</p> Full article ">Figure 14
<p>Comparison of continuity [tan(<span class="html-italic">d_y</span>/<span class="html-italic">d_x</span>)] between estimated position points.</p> Full article ">Figure 15
<p>Comparison of steering angle command of path-following controller.</p> Full article ">

16 pages, 3098 KiB  

Open AccessTechnical Note

Compressed Sensing Imaging with Compensation of Motion Errors for MIMO Radar

by Haoran Li, Shuangxun Li, Zhi Li, Yongpeng Dai and Tian Jin

Remote Sens. 2021, 13(23), 4909; https://doi.org/10.3390/rs13234909 - 3 Dec 2021

Cited by 4 | Viewed by 2215

Abstract

Using a multiple-input-multiple-output (MIMO) radar for environment sensing is gaining more attention in unmanned ground vehicles (UGV). During the movement of the UGV, the position of MIMO array compared to the ideal imaging position will inevitably change. Although compressed sensing (CS) imaging can [...] Read more.

Using a multiple-input-multiple-output (MIMO) radar for environment sensing is gaining more attention in unmanned ground vehicles (UGV). During the movement of the UGV, the position of MIMO array compared to the ideal imaging position will inevitably change. Although compressed sensing (CS) imaging can provide high resolution imaging results and reduce the complexity of the system, the inaccurate MIMO array elements position will lead to defocusing of imaging. In this paper, a method is proposed to realize MIMO array motion error compensation and sparse imaging simultaneously. It utilizes a block coordinate descent (BCD) method, which iteratively estimates the motion errors of the transmitting and receiving elements, as well as synchronously achieving the autofocus imaging. The method accurately estimates and compensates for the motion errors of the transmitters and receivers, rather than approximating them as phase errors in the data. The validity of the proposed method is verified by simulation and measured experiments in a smoky environment. Full article

(This article belongs to the Topic Information Sensing Technology for Intelligent/Driverless Vehicle)

►▼ Show Figures

Figure 1

Figure 1
<p>Imaging geometry of MIMO Radar.</p> Full article ">Figure 2
<p>Flow chart of Algorithm 1.</p> Full article ">Figure 3
<p>Imaging Results Contrast. (<b>a</b>) Results without compensation of errors; (<b>b</b>) Results of the proposed method.</p> Full article ">Figure 4
<p>MIMO array motion errors estimation. (<b>a</b>) <math display="inline"><semantics> <mi>x</mi> </semantics></math> dimension of Transmitters. (<b>b</b>) <math display="inline"><semantics> <mi>y</mi> </semantics></math> dimension of Transmitters. (<b>c</b>) <math display="inline"><semantics> <mi>z</mi> </semantics></math> dimension of Transmitters. (<b>d</b>) <math display="inline"><semantics> <mi>x</mi> </semantics></math> dimension of Receivers. (<b>e</b>) <math display="inline"><semantics> <mi>y</mi> </semantics></math> dimension of Receivers. (<b>f</b>) <math display="inline"><semantics> <mi>z</mi> </semantics></math> dimension of Receivers.</p> Full article ">Figure 5
<p>Data error, Reconstruction error, and RMSE of <math display="inline"><semantics> <mrow> <msub> <mstyle mathvariant="bold" mathsize="normal"> <mi>e</mi> </mstyle> <mi>t</mi> </msub> </mrow> </semantics></math> and <math display="inline"><semantics> <mrow> <msub> <mstyle mathvariant="bold" mathsize="normal"> <mi>e</mi> </mstyle> <mi>r</mi> </msub> </mrow> </semantics></math> across the iteration. (<b>a</b>) Data error. (<b>b</b>) Reconstruction error. (<b>c</b>) RMSE of <math display="inline"><semantics> <mrow> <msub> <mstyle mathvariant="bold" mathsize="normal"> <mi>e</mi> </mstyle> <mi>t</mi> </msub> </mrow> </semantics></math>. (<b>d</b>) RMSE of <math display="inline"><semantics> <mrow> <msub> <mstyle mathvariant="bold" mathsize="normal"> <mi>e</mi> </mstyle> <mi>r</mi> </msub> </mrow> </semantics></math>.</p> Full article ">Figure 6
<p>Proposed method performance under different SNRs. (<b>a</b>) RMSE of the average of six directions motion errors. (<b>b</b>) Target reconstruction error. (<b>c</b>) Data error.</p> Full article ">Figure 7
<p>Optical images of experimental scenes. (<b>a</b>) MIMO radar mounted on the UGV (<b>b</b>) UGV in the fog. (<b>c</b>) Diagram of corner reflectors distribution. (<b>d</b>) Optical images of three corner reflectors in a smoky scene.</p> Full article ">Figure 8
<p>MIMO radar experiments results. (<b>a</b>) Result of BP. (<b>b</b>) Result without compensation of errors. (<b>c</b>) Result of the proposed method.</p> Full article ">

17 pages, 6413 KiB  

Open AccessArticle

A Distance Increment Smoothing Method and Its Application on the Detection of NLOS in the Cooperative Positioning

by Dongqing Zhao, Dongmin Wang, Minzhi Xiang, Jinfei Li, Chaoyong Yang, Letian Zhang and Linyang Li

Sensors 2021, 21(23), 8028; https://doi.org/10.3390/s21238028 - 1 Dec 2021

Viewed by 2002

Abstract

The wide use of cooperative missions using multiple unmanned platforms has made relative distance information an essential factor for cooperative positioning and formation control. Reducing the range error effectively in real time has become the main technical challenge. We present a new method [...] Read more.

The wide use of cooperative missions using multiple unmanned platforms has made relative distance information an essential factor for cooperative positioning and formation control. Reducing the range error effectively in real time has become the main technical challenge. We present a new method to deal with ranging errors based on the distance increment (DI). The DI calculated by dead reckoning is used to smooth the DI obtained by the cooperative positioning, and the smoothed DI is then used to detect and estimate the non-line-of-sight (NLOS) error as well as to smooth the observed values containing random noise in the filtering process. Simulation and experimental results show that the relative accuracy of NLOS estimation is 8.17%, with the maximum random error reduced by 40.27%. The algorithm weakens the influence of NLOS and random errors on the measurement distance, thus improving the relative distance precision and enhancing the stability and reliability of cooperative positioning. Full article

(This article belongs to the Topic Information Sensing Technology for Intelligent/Driverless Vehicle)

►▼ Show Figures

Figure 1

Figure 1
<p>Structure of cooperative positioning algorithm.</p> Full article ">Figure 2
<p>DI error of cooperative positioning (blue dashed) and dead reckoning (red solid).</p> Full article ">Figure 3
<p>Distance error of cooperative positioning (red solid) and dead reckoning (bule dashed).</p> Full article ">Figure 4
<p>The process of NLOS dynamic estimation.</p> Full article ">Figure 5
<p>UWB ranging error processing process.</p> Full article ">Figure 6
<p>Experimental equipment.</p> Full article ">Figure 7
<p>Trajectories of three platforms.</p> Full article ">Figure 8
<p>DI error of cooperative positioning (blue dashed), dead reckoning (orange solid), and estimation (green dashdotted).</p> Full article ">Figure 9
<p>Distance errors of observed distance (yellow dashed), distance adding NLOS (blue solid), RN–Based distance (green densely dotted), and DI–Based distance (orange dashdotted).</p> Full article ">Figure 10
<p>The effect of step size on the three NLOS estimates: (<b>a</b>) NLOS1, (<b>b</b>) NLOS2, and (<b>c</b>) NLOS3.</p> Full article ">Figure 11
<p>Distance errors of the observed distance (blue solid), RN–Based distance (green dashed), and DI–Based distance (orange dashdotted) over different segments: (<b>a</b>) A–B, (<b>b</b>) B–C, and (<b>c</b>) A–C.</p> Full article ">Figure 12
<p>Obtained positioning errors of (<b>a</b>) Platform A and (<b>b</b>) Platform C using observed distance (blue solid), RN–Based distance (green dashed), and DI–Based distance (orange dashdotted).</p> Full article ">

24 pages, 24794 KiB  

Open AccessArticle

A Novel Anti-Drift Visual Object Tracking Algorithm Based on Sparse Response and Adaptive Spatial-Temporal Context-Aware

by Yinqiang Su, Jinghong Liu, Fang Xu, Xueming Zhang and Yujia Zuo

Remote Sens. 2021, 13(22), 4672; https://doi.org/10.3390/rs13224672 - 19 Nov 2021

Cited by 7 | Viewed by 2594

Abstract

Correlation filter (CF) based trackers have gained significant attention in the field of visual single-object tracking, owing to their favorable performance and high efficiency; however, existing trackers still suffer from model drift caused by boundary effects and filter degradation. In visual tracking, long-term [...] Read more.

Correlation filter (CF) based trackers have gained significant attention in the field of visual single-object tracking, owing to their favorable performance and high efficiency; however, existing trackers still suffer from model drift caused by boundary effects and filter degradation. In visual tracking, long-term occlusion and large appearance variations easily cause model degradation. To remedy these drawbacks, we propose a sparse adaptive spatial-temporal context-aware method that effectively avoids model drift. Specifically, a global context is explicitly incorporated into the correlation filter to mitigate boundary effects. Subsequently, an adaptive temporal regularization constraint is adopted in the filter training stage to avoid model degradation. Meanwhile, a sparse response constraint is introduced to reduce the risk of further model drift. Furthermore, we apply the alternating direction multiplier method (ADMM) to derive a closed-solution of the object function with a low computational cost. In addition, an updating scheme based on the APCE-pool and Peak-pool is proposed to reveal the tracking condition and ensure updates of the target’s appearance model with high-confidence. The Kalam filter is adopted to track the target when the appearance model is persistently unreliable and abnormality occurs. Finally, extensive experimental results on OTB-2013, OTB-2015 and VOT2018 datasets show that our proposed tracker performs favorably against several state-of-the-art trackers. Full article

(This article belongs to the Topic Information Sensing Technology for Intelligent/Driverless Vehicle)

►▼ Show Figures

Figure 1

Figure 1
<p>Flow chart of the Ad_SASTCA framework.</p> Full article ">Figure 2
<p>The APCE and peak normalized change curve of our Ad_SASTCA model on the sequence “Liquor” from 1200 frame to 1380 frame. Severe occlusion in frames 1237, 1287, 1230 and 1356 nearby will lead the APCE and peak of response to dropping sharply in a short period. Because the Ad_SASTCA does not fuse excessive incorrect information into the appearance model, the response map still indicates a sharp unimodal peak, and the model drift is avoided.</p> Full article ">Figure 3
<p>Overall experimental results of precision plot (<b>a</b>) and success plot (<b>b</b>) comparing Ad_SASTCA with nine state-of the art trackers on OTB-2013. The trackers in figures (<b>a</b>) and (<b>b</b>) are sorted according to precision and AUCs, respectively. Best viewed in color and high resolution.</p> Full article ">Figure 4
<p>Overall results of precision plot (<b>a</b>) and success plot (<b>b</b>) comparing Ad_SASTCA with nine state-of the art trackers on OTB-2015 dataset. The trackers in figures (<b>a</b>) and (<b>b</b>) are sorted according to precision and AUCs, respectively.</p> Full article ">Figure 5
<p>Attribute based evaluation. Precision plots (<b>a</b>–<b>k</b>) are indicated on the OTB-2015 dataset for 11 challenge attributes. Precision rates are reported in brackets. The title of each plot includes the number of videos related attributes.</p> Full article ">Figure 6
<p>Illustration of qualitative tracking results on challenging sequences (From top to bottom: (<b>a</b>) lemming, (<b>b</b>) BlurOwl, (<b>c</b>) Girl2, (<b>d</b>) Bird1, (<b>e</b>) Shaking, (<b>f</b>) Board, (<b>g</b>) Human2 and (<b>h</b>) Human3). The colour bounding boxes are the corresponding results of Ad_SASTCA, STAPLE_CA [<a href="#B28-remotesensing-13-04672" class="html-bibr">28</a>], CSR_DCF [<a href="#B32-remotesensing-13-04672" class="html-bibr">32</a>], SRDCF [<a href="#B26-remotesensing-13-04672" class="html-bibr">26</a>] and AutoTrack [<a href="#B34-remotesensing-13-04672" class="html-bibr">34</a>].</p> Full article ">Figure 6 Cont.
<p>Illustration of qualitative tracking results on challenging sequences (From top to bottom: (<b>a</b>) lemming, (<b>b</b>) BlurOwl, (<b>c</b>) Girl2, (<b>d</b>) Bird1, (<b>e</b>) Shaking, (<b>f</b>) Board, (<b>g</b>) Human2 and (<b>h</b>) Human3). The colour bounding boxes are the corresponding results of Ad_SASTCA, STAPLE_CA [<a href="#B28-remotesensing-13-04672" class="html-bibr">28</a>], CSR_DCF [<a href="#B32-remotesensing-13-04672" class="html-bibr">32</a>], SRDCF [<a href="#B26-remotesensing-13-04672" class="html-bibr">26</a>] and AutoTrack [<a href="#B34-remotesensing-13-04672" class="html-bibr">34</a>].</p> Full article ">Figure 7
<p>The performance of different components and their combinations in Ad_SASTCA, evaluated on OTB-2013. The trackers in figures (<b>a</b>) and (<b>b</b>) are sorted according to precision and AUCs, respectively.</p> Full article ">

21 pages, 5253 KiB  

Open AccessArticle

An Overdispersed Black-Box Variational Bayesian–Kalman Filter with Inaccurate Noise Second-Order Statistics

by Lin Cao, Chuyuan Zhang, Zongmin Zhao, Dongfeng Wang, Kangning Du, Chong Fu and Jianfeng Gu

Sensors 2021, 21(22), 7673; https://doi.org/10.3390/s21227673 - 18 Nov 2021

Viewed by 2106

Abstract

Aimed at the problems in which the performance of filters derived from a hypothetical model will decline or the cost of time of the filters derived from a posterior model will increase when prior knowledge and second-order statistics of noise are uncertain, a [...] Read more.

Aimed at the problems in which the performance of filters derived from a hypothetical model will decline or the cost of time of the filters derived from a posterior model will increase when prior knowledge and second-order statistics of noise are uncertain, a new filter is proposed. In this paper, a Bayesian robust Kalman filter based on posterior noise statistics (KFPNS) is derived, and the recursive equations of this filter are very similar to that of the classical algorithm. Note that the posterior noise distributions are approximated by overdispersed black-box variational inference (O-BBVI). More precisely, we introduce an overdispersed distribution to push more probability density to the tails of variational distribution and incorporated the idea of importance sampling into two strategies of control variates and Rao–Blackwellization in order to reduce the variance of estimators. As a result, the convergence process will speed up. From the simulations, we can observe that the proposed filter has good performance for the model with uncertain noise. Moreover, we verify the proposed algorithm by using a practical multiple-input multiple-output (MIMO) radar system. Full article

(This article belongs to the Topic Information Sensing Technology for Intelligent/Driverless Vehicle)

►▼ Show Figures

Figure 1

Figure 1
<p>KFPNS framework.</p> Full article ">Figure 2
<p>The principle of conversion.</p> Full article ">Figure 3
<p>Average MSE of various filters when <math display="inline"><semantics> <mrow> <msub> <mi>α</mi> <mn>1</mn> </msub> </mrow> </semantics></math> and <math display="inline"><semantics> <mrow> <msub> <mi>α</mi> <mn>2</mn> </msub> </mrow> </semantics></math> are uncertain.</p> Full article ">Figure 4
<p>Performance analysis for specific noise pairs. (<b>a</b>) The average MSE for specific noise model <math display="inline"><semantics> <mrow> <msub> <mi>α</mi> <mn>1</mn> </msub> <mo>=</mo> <mn>3</mn> <mo>,</mo> <msub> <mi>α</mi> <mn>2</mn> </msub> <mo>=</mo> <mn>1.5</mn> </mrow> </semantics></math>. (<b>b</b>) The average MSE for specific noise model <math display="inline"><semantics> <mrow> <msub> <mi>α</mi> <mn>1</mn> </msub> <mo>=</mo> <mn>3.5</mn> <mo>,</mo> <msub> <mi>α</mi> <mn>2</mn> </msub> <mo>=</mo> <mn>3.5</mn> </mrow> </semantics></math>. (<b>c</b>) The variation of <math display="inline"><semantics> <mrow> <mi mathvariant="double-struck">E</mi> <mrow> <mo>[</mo> <mrow> <msub> <mi>α</mi> <mn>1</mn> </msub> <mo>|</mo> <msub> <mi mathvariant="script">Y</mi> <mi>k</mi> </msub> </mrow> <mo>]</mo> </mrow> </mrow> </semantics></math> and its variance when <math display="inline"><semantics> <mrow> <msub> <mi>α</mi> <mn>1</mn> </msub> <mo>=</mo> <mn>3</mn> </mrow> </semantics></math>. (<b>d</b>) The variation of <math display="inline"><semantics> <mrow> <mi mathvariant="double-struck">E</mi> <mrow> <mo>[</mo> <mrow> <msub> <mi>α</mi> <mn>1</mn> </msub> <mo>|</mo> <msub> <mi mathvariant="script">Y</mi> <mi>k</mi> </msub> </mrow> <mo>]</mo> </mrow> </mrow> </semantics></math> and its variance when <math display="inline"><semantics> <mrow> <msub> <mi>α</mi> <mn>1</mn> </msub> <mo>=</mo> <mn>3.5</mn> </mrow> </semantics></math>. (<b>e</b>) The variation of <math display="inline"><semantics> <mrow> <mi mathvariant="double-struck">E</mi> <mrow> <mo>[</mo> <mrow> <msub> <mi>α</mi> <mn>2</mn> </msub> <mo>|</mo> <msub> <mi mathvariant="script">Y</mi> <mi>k</mi> </msub> </mrow> <mo>]</mo> </mrow> </mrow> </semantics></math> and its variance when <math display="inline"><semantics> <mrow> <msub> <mi>α</mi> <mn>2</mn> </msub> <mo>=</mo> <mn>1.5</mn> </mrow> </semantics></math>. (<b>f</b>) The variation of <math display="inline"><semantics> <mrow> <mi mathvariant="double-struck">E</mi> <mrow> <mo>[</mo> <mrow> <msub> <mi>α</mi> <mn>2</mn> </msub> <mo>|</mo> <msub> <mi mathvariant="script">Y</mi> <mi>k</mi> </msub> </mrow> <mo>]</mo> </mrow> </mrow> </semantics></math> and its variance when <math display="inline"><semantics> <mrow> <msub> <mi>α</mi> <mn>2</mn> </msub> <mo>=</mo> <mn>3.5</mn> </mrow> </semantics></math>.</p> Full article ">Figure 5
<p>The average performance of four filters with different Beta priors. (<b>a</b>) <math display="inline"><semantics> <mrow> <msub> <mi>α</mi> <mi>r</mi> </msub> <mo>=</mo> <mn>0.1</mn> <mo>,</mo> <msub> <mi>β</mi> <mi>r</mi> </msub> <mo>=</mo> <mn>0.9</mn> </mrow> </semantics></math>; (<b>b</b>) <math display="inline"><semantics> <mrow> <msub> <mi>α</mi> <mi>r</mi> </msub> <mo>=</mo> <mn>0.01</mn> <mo>,</mo> <msub> <mi>β</mi> <mi>r</mi> </msub> <mo>=</mo> <mn>0.09</mn> </mrow> </semantics></math>.</p> Full article ">Figure 6
<p>Average MSE of various filters when <math display="inline"><semantics> <mrow> <msub> <mi>α</mi> <mn>2</mn> </msub> </mrow> </semantics></math> is uncertain, and the interval <math display="inline"><semantics> <mrow> <mo stretchy="false">[</mo> <mn>3</mn> <mo>,</mo> <mn>5</mn> <mo stretchy="false">]</mo> </mrow> </semantics></math> of prior distribution is inaccurate. (<b>a</b>) The exact interval is <math display="inline"><semantics> <mrow> <mo stretchy="false">[</mo> <mn>2</mn> <mo>,</mo> <mn>6</mn> <mo stretchy="false">]</mo> </mrow> </semantics></math>. (<b>b</b>) The exact interval is <math display="inline"><semantics> <mrow> <mo stretchy="false">[</mo> <mn>0.5</mn> <mo>,</mo> <mn>7.5</mn> <mo stretchy="false">]</mo> </mrow> </semantics></math>.</p> Full article ">Figure 7
<p>Real data acquisition scenes. (<b>a</b>) Indoor scene. (<b>b</b>) Outdoor scene.</p> Full article ">Figure 8
<p>The fitting degree between the estimated trajectory and the true trajectory after processing by various Kalman filtering methods in an indoor scene. (<b>a</b>) OBKF. (<b>b</b>) KFPNS. (<b>c</b>) IBRKF. (<b>d</b>) CKF.</p> Full article ">Figure 9
<p>The fitting degree between the estimated trajectory and the true trajectory after processing by various Kalman filtering methods in an outdoor scene. (<b>a</b>) OBKF. (<b>b</b>) KFPNS. (<b>c</b>) IBRKF. (<b>d</b>) CKF.</p> Full article ">Figure 10
<p>The PE of each filter in two specific scenes. (<b>a</b>) Comparison of PE of various Kalman filters in an indoor scene. (<b>b</b>) Comparison of PE of various Kalman filters in an outdoor scene.</p> Full article ">Figure 11
<p>The RMSE of various Kalman filtering methods in different scenes. (<b>a</b>) The RMSE of the four Kalman filtering methods in an indoor scene with respect to target A and target B, respectively. (<b>b</b>) The RMSE of the four Kalman filtering methods in an outdoor scene with respect to target A and target B, respectively.</p> Full article ">Figure 12
<p>CDFs of RMSE for various algorithms in an indoor scene.</p> Full article ">Figure 13
<p>The time consumption of two algorithms when the length of the observation sequence changes.</p> Full article ">

27 pages, 8861 KiB  

Open AccessArticle

Adaptive Fast Non-Singular Terminal Sliding Mode Path Following Control for an Underactuated Unmanned Surface Vehicle with Uncertainties and Unknown Disturbances

by Yunsheng Fan, Bowen Liu, Guofeng Wang and Dongdong Mu

Sensors 2021, 21(22), 7454; https://doi.org/10.3390/s21227454 - 10 Nov 2021

Cited by 13 | Viewed by 2865

Abstract

This paper focuses on an issue involving robust adaptive path following for the uncertain underactuated unmanned surface vehicle with time-varying large sideslips angle and actuator saturation. An improved line-of-sight guidance law based on a reduced-order extended state observer is proposed to address the [...] Read more.

This paper focuses on an issue involving robust adaptive path following for the uncertain underactuated unmanned surface vehicle with time-varying large sideslips angle and actuator saturation. An improved line-of-sight guidance law based on a reduced-order extended state observer is proposed to address the large sideslip angle that occurs in practical navigation. Next, the finite-time disturbances observer is designed by considering the perturbations parameter of the model and the unknown disturbances of the external environment as the lumped disturbances. Then, an adaptive term is introduced into Fast Non-singular Terminal Sliding Mode Control to design the path following controllers. Finally, considering the saturation of actuator, an auxiliary dynamic system is introduced. By selecting the appropriate design parameters, all the signals of the whole path following a closed-loop system can be ultimately bounded. Real-time control of path following can be achieved by transferring data from shipborne sensors such as GPS, combined inertial guidance and anemoclinograph to the Fast Non-singular Terminal Sliding Mode controller. Two examples as comparisons were carried out to demonstrate the validity of the proposed control approach. Full article

(This article belongs to the Topic Information Sensing Technology for Intelligent/Driverless Vehicle)

►▼ Show Figures

Figure 1

Figure 1
<p>Schematic diagram of USV path-following guidance.</p> Full article ">Figure 2
<p>The Block Diagram of The Path Following Controller.</p> Full article ">Figure 3
<p>The Shipborne Sensors for “Lanxin”.</p> Full article ">Figure 4
<p>Comparison results of straight line trajectory tracking at moderate speed.</p> Full article ">Figure 5
<p>Along-track error <math display="inline"><semantics> <msub> <mi>x</mi> <mi>e</mi> </msub> </semantics></math> and cross-track error <math display="inline"><semantics> <msub> <mi>y</mi> <mi>e</mi> </msub> </semantics></math> at middle speed.</p> Full article ">Figure 6
<p>Sideslip angle estimations at moderate speed.</p> Full article ">Figure 7
<p>Comparison results of <math display="inline"><semantics> <msub> <mi>u</mi> <mi>e</mi> </msub> </semantics></math> and <math display="inline"><semantics> <msub> <mi>ψ</mi> <mi>e</mi> </msub> </semantics></math> at moderate speed.</p> Full article ">Figure 8
<p>The lumped disturbances and their estimations at moderate speed.</p> Full article ">Figure 9
<p>The force <math display="inline"><semantics> <msub> <mi>τ</mi> <mi>u</mi> </msub> </semantics></math> and moment <math display="inline"><semantics> <msub> <mi>τ</mi> <mi>r</mi> </msub> </semantics></math> at moderate speed.</p> Full article ">Figure 10
<p>Comparison results of straight line trajectory tracking at fast speed.</p> Full article ">Figure 11
<p>Along-track error <math display="inline"><semantics> <msub> <mi>x</mi> <mi>e</mi> </msub> </semantics></math> and cross-track error <math display="inline"><semantics> <msub> <mi>y</mi> <mi>e</mi> </msub> </semantics></math> at fast speed.</p> Full article ">Figure 12
<p>Sideslip angle estimations at fast speed.</p> Full article ">Figure 13
<p>Comparison results of <math display="inline"><semantics> <msub> <mi>u</mi> <mi>e</mi> </msub> </semantics></math> and <math display="inline"><semantics> <msub> <mi>ψ</mi> <mi>e</mi> </msub> </semantics></math> at fast speed.</p> Full article ">Figure 14
<p>The lumped disturbances and their estimations at fast speed.</p> Full article ">Figure 15
<p>The force <math display="inline"><semantics> <msub> <mi>τ</mi> <mi>u</mi> </msub> </semantics></math> and moment <math display="inline"><semantics> <msub> <mi>τ</mi> <mi>r</mi> </msub> </semantics></math> at fast speed.</p> Full article ">Figure 16
<p>Comparison results of curve line trajectory tracking at moderate speed.</p> Full article ">Figure 17
<p>Along-track error <math display="inline"><semantics> <msub> <mi>x</mi> <mi>e</mi> </msub> </semantics></math> and cross-track error <math display="inline"><semantics> <msub> <mi>y</mi> <mi>e</mi> </msub> </semantics></math> at moderate speed.</p> Full article ">Figure 18
<p>Sideslip angle estimations at moderate speed.</p> Full article ">Figure 19
<p>Comparison results of <math display="inline"><semantics> <msub> <mi>u</mi> <mi>e</mi> </msub> </semantics></math> and <math display="inline"><semantics> <msub> <mi>ψ</mi> <mi>e</mi> </msub> </semantics></math> at moderate speed.</p> Full article ">Figure 20
<p>The lumped disturbances and their estimations at moderate speed.</p> Full article ">Figure 21
<p>The force <math display="inline"><semantics> <msub> <mi>τ</mi> <mi>u</mi> </msub> </semantics></math> and moment <math display="inline"><semantics> <msub> <mi>τ</mi> <mi>r</mi> </msub> </semantics></math> at moderate speed.</p> Full article ">Figure 22
<p>Comparison results of curve line trajectory tracking at fast speed.</p> Full article ">Figure 23
<p>Along-track error <math display="inline"><semantics> <msub> <mi>x</mi> <mi>e</mi> </msub> </semantics></math> and cross-track error <math display="inline"><semantics> <msub> <mi>y</mi> <mi>e</mi> </msub> </semantics></math> at fast speed.</p> Full article ">Figure 24
<p>Sideslip angle estimations at fast speed.</p> Full article ">Figure 25
<p>Comparison results of <math display="inline"><semantics> <msub> <mi>u</mi> <mi>e</mi> </msub> </semantics></math> and <math display="inline"><semantics> <msub> <mi>ψ</mi> <mi>e</mi> </msub> </semantics></math> at fast speed.</p> Full article ">Figure 26
<p>The lumped disturbances and their estimations at fast speed.</p> Full article ">Figure 27
<p>The force <math display="inline"><semantics> <msub> <mi>τ</mi> <mi>u</mi> </msub> </semantics></math> and moment <math display="inline"><semantics> <msub> <mi>τ</mi> <mi>r</mi> </msub> </semantics></math> at fast speed.</p> Full article ">Figure 28
<p>Comparison results of curve line trajectory tracking under severe disturbance.</p> Full article ">Figure 29
<p>Along-track error <math display="inline"><semantics> <msub> <mi>x</mi> <mi>e</mi> </msub> </semantics></math> and cross-track error <math display="inline"><semantics> <msub> <mi>y</mi> <mi>e</mi> </msub> </semantics></math> under severe disturbance.</p> Full article ">Figure 30
<p>Sideslip angle estimations under severe disturbance.</p> Full article ">Figure 31
<p>Comparison results of <math display="inline"><semantics> <msub> <mi>u</mi> <mi>e</mi> </msub> </semantics></math> and <math display="inline"><semantics> <msub> <mi>ψ</mi> <mi>e</mi> </msub> </semantics></math> under severe disturbance.</p> Full article ">Figure 32
<p>The lumped disturbances and their estimations under severe disturbance.</p> Full article ">Figure 33
<p>The force <math display="inline"><semantics> <msub> <mi>τ</mi> <mi>u</mi> </msub> </semantics></math> and moment <math display="inline"><semantics> <msub> <mi>τ</mi> <mi>r</mi> </msub> </semantics></math> under severe disturbance.</p> Full article ">

24 pages, 3297 KiB  

Open AccessReview

Deep Neural Networks to Detect Weeds from Crops in Agricultural Environments in Real-Time: A Review

by Ildar Rakhmatulin, Andreas Kamilaris and Christian Andreasen

Remote Sens. 2021, 13(21), 4486; https://doi.org/10.3390/rs13214486 - 8 Nov 2021

Cited by 68 | Viewed by 16706

Abstract

Automation, including machine learning technologies, are becoming increasingly crucial in agriculture to increase productivity. Machine vision is one of the most popular parts of machine learning and has been widely used where advanced automation and control have been required. The trend has shifted [...] Read more.

Automation, including machine learning technologies, are becoming increasingly crucial in agriculture to increase productivity. Machine vision is one of the most popular parts of machine learning and has been widely used where advanced automation and control have been required. The trend has shifted from classical image processing and machine learning techniques to modern artificial intelligence (AI) and deep learning (DL) methods. Based on large training datasets and pre-trained models, DL-based methods have proven to be more accurate than previous traditional techniques. Machine vision has wide applications in agriculture, including the detection of weeds and pests in crops. Variation in lighting conditions, failures to transfer learning, and object occlusion constitute key challenges in this domain. Recently, DL has gained much attention due to its advantages in object detection, classification, and feature extraction. DL algorithms can automatically extract information from large amounts of data used to model complex problems and is, therefore, suitable for detecting and classifying weeds and crops. We present a systematic review of AI-based systems to detect weeds, emphasizing recent trends in DL. Various DL methods are discussed to clarify their overall potential, usefulness, and performance. This study indicates that several limitations obstruct the widespread adoption of AI/DL in commercial applications. Recommendations for overcoming these challenges are summarized. Full article

(This article belongs to the Topic Information Sensing Technology for Intelligent/Driverless Vehicle)

►▼ Show Figures

Graphical abstract

Graphical abstract
Full article ">Figure 1
<p>Examples of weed images used for transfer learning.</p> Full article ">Figure 2
<p>Examples of images from the CropDeep dataset.</p> Full article ">Figure 3
<p>Examples of artificial images generated by LASSR [<a href="#B68-remotesensing-13-04486" class="html-bibr">68</a>].</p> Full article ">Figure 4
<p>Image processing of pigweed (<span class="html-italic">A. retroflexus</span>): (<b>a</b>) original image, (<b>b</b>) grey-scale image, (<b>c</b>) after sharpening, (<b>d</b>) with noise filter [<a href="#B77-remotesensing-13-04486" class="html-bibr">77</a>].</p> Full article ">Figure 5
<p>Example of auto-calibration process: (<b>a</b>) original image, (<b>b</b>) corrected image [<a href="#B64-remotesensing-13-04486" class="html-bibr">64</a>].</p> Full article ">Figure 6
<p>Example of a robot for removing weeds [<a href="#B85-remotesensing-13-04486" class="html-bibr">85</a>].</p> Full article ">Figure 7
<p>Detection of objects within dense scenes: (<b>a</b>) for apples [<a href="#B99-remotesensing-13-04486" class="html-bibr">99</a>], (<b>b</b>) canola and weeds (initial image) [<a href="#B100-remotesensing-13-04486" class="html-bibr">100</a>], (<b>c</b>) weeds after semantic segmentation [<a href="#B100-remotesensing-13-04486" class="html-bibr">100</a>].</p> Full article ">Figure 8
<p>Counting of corn kernels processed in images with CNN [<a href="#B111-remotesensing-13-04486" class="html-bibr">111</a>].</p> Full article ">Figure A1
<p>Block diagram on the various stages of the weed detection process. Numbers refer to related subsections in the article.</p> Full article ">

13 pages, 2171 KiB  

Open AccessArticle

A Robust and Fast Method for Sidescan Sonar Image Segmentation Based on Region Growing

by Xuyang Wang, Luyu Wang, Guolin Li and Xiang Xie

Sensors 2021, 21(21), 6960; https://doi.org/10.3390/s21216960 - 20 Oct 2021

Cited by 14 | Viewed by 3278

Abstract

For high-resolution side scan sonar images, accurate and fast segmentation of sonar images is crucial for underwater target detection and recognition. However, due to the characteristics of low signal-to-noise ratio (SNR) and complex environmental noise of sonar, the existing methods with [...] Read more.

For high-resolution side scan sonar images, accurate and fast segmentation of sonar images is crucial for underwater target detection and recognition. However, due to the characteristics of low signal-to-noise ratio (SNR) and complex environmental noise of sonar, the existing methods with high accuracy and good robustness are mostly iterative methods with high complexity and poor real-time performance. For this purpose, a region growing based segmentation using the likelihood ratio testing method (RGLT) is proposed. This method obtains the seed points in the highlight and the shadow regions by likelihood ratio testing based on the statistical probability distribution and then grows them according to the similarity criterion. The growth avoids the processing of the seabed reverberation regions, which account for the largest proportion of sonar images, thus greatly reducing segmentation time and improving segmentation accuracy. In addition, a pre-processing filtering method called standard deviation filtering (STDF) is proposed to improve the SNR and remove the speckle noise. Experiments were conducted on three sonar databases, which showed that RGLT has significantly improved quantitative metrics such as accuracy, speed, and segmentation visual effects. The average accuracy and running times of the proposed segmentation method for 100 × 400 images are separately 95.90% and 0.44 s. Full article

(This article belongs to the Topic Information Sensing Technology for Intelligent/Driverless Vehicle)

►▼ Show Figures

Figure 1

Figure 1
<p>Block diagram of the proposed method process.</p> Full article ">Figure 2
<p>Comparison of a sonar image with and without time gain re-compensation. (<b>a</b>) A sonar image without time gain re-compensation. (<b>b</b>) A sonar image with time gain re-compensation. (<b>c</b>) The mean of the intensity at range R for origin data and re-compensated data.</p> Full article ">Figure 3
<p>Results of the different filters. The subgraph (<b>a</b>) shows a raw SSS image without filtering, and subgraph (<b>b</b>–<b>f</b>), respectively, show the results for mean filtering, median filtering, Gaussian filtering, Lee filtering, and the <span class="html-italic">STDF</span> proposed in this paper.</p> Full article ">Figure 4
<p>Results of different filters for downsampled sonar images. The column downsampling factors for subgraph (<b>A</b>), subgraph (<b>B</b>), and subgraph (<b>C</b>) are 2, 4, and 8, respectively. (<b>a</b>) Raw SSS image. (<b>b</b>) Results for mean filtering. (<b>c</b>) Results for median filtering. (<b>d</b>) Results for Gaussian filtering (<b>e</b>) Results for Lee filtering. (<b>f</b>) Results for proposed <span class="html-italic">STDF</span>.</p> Full article ">Figure 5
<p>Segmentation results of the four methods: (<b>a</b>) The real sonar images. (<b>b</b>) The corresponding manually labeled ground truth segmented images. The green areas represent the shadow areas, and the red areas represent the highlighted areas. (<b>c</b>) The segmentation results of fuzzy C-means. (<b>d</b>) The segmentation results of the active contour. (<b>e</b>) The segmentation results of RGLT I. (<b>f</b>) The segmentation results of RGLT II.</p> Full article ">

Show export options expand_more Show export options expand_less

Select all

Export citation of selected articles as:

Plain Text BibTeX BibTeX (without abstracts) Endnote Endnote (without abstracts) Tab-delimited RIS

 

Error

Oops... you haven't selected anything for export.

Ok

clear

Displaying articles 1-29