Search Results (316)

Several studies have used aerial images to predict physiological maturity (R8 stage) in soybeans (Glycine max (L.) Merr.). However, information for making predictions in the current growing season using models fitted in previous years is still necessary. Using the Random Forest machine learning algorithm and time series of RGB (red, green, blue) and multispectral images taken from a drone, this work aimed to study, in three breeding experiments of plant rows, how maturity predictions are impacted by a number of factors. These include the type of camera used, the number and time between flights, and whether models fitted with data obtained in one or more environments can be used to make accurate predictions in an independent environment. Applying principal component analysis (PCA), it was found that compared to the full set of 8–10 flights (R² = 0.91–0.94; RMSE = 1.8–1.3 days), using data from three to five fights before harvest had almost no effect on the prediction error (RMSE increase ~0.1 days). Similar prediction accuracy was achieved using either a multispectral or an affordable RGB camera, and the excess green index (ExG) was found to be the important feature in making predictions. Using a model trained with data from two previous years and using fielding notes from check cultivars planted in the test season, the R8 stage was predicted, in 2020, with an error of 2.1 days. Periodically adjusted models could help soybean breeding programs save time when characterizing the cycle length of thousands of plant rows each season. Full article

The use of UAVs (drones) and photogrammetry has gained attention in recent years in the construction industry, allowing information to be obtained from a given area without having direct contact with the area, and thus, being a more efficient alternative in terms of time and costs when compared to a traditional topographic survey. Due to the increase in the use of UAVs for photogrammetry, an investigation is proposed to determine the influence of a non-controllable component in photogrammetric flights: the weather. Factors such as brightness, temperature, wind, KP index, and solar radiation affect the precision and quality of the images to be used in photogrammetry. This research determines which factors are most influential in these results through a varied database obtained over a year. In this way, the moments with the most favorable conditions for a photogrammetric flight in climates such as that of the city of Lima or similar are established. A total of 448 flights carried out over a year were analyzed, collecting climatic data such as air temperature, speed and wind direction, solar radiation, and KP index. The flights, which were carried out with a Mavic 2 Pro UAV, were carried out at 100 m high and with a camera at 90° to obtain detailed information on the works. Full article

In recent years, unmanned aerial vehicles (UAVs) have shown significant potential across diverse applications, drawing attention from both academia and industry. In specific scenarios, UAVs are expected to achieve formation flying without relying on communication or external assistance. In this context, our work focuses on the classic leader-follower formation and presents a learning-based UAV chasing control method that enables a quadrotor UAV to autonomously chase a highly maneuverable fixed-wing UAV. The proposed method utilizes a neural network called Vision Follow Net (VFNet), which integrates monocular visual data with the UAV’s flight state information. Utilizing a multi-head self-attention mechanism, VFNet aggregates data over a time window to predict the waypoints for the chasing flight. The quadrotor’s yaw angle is controlled by calculating the line-of-sight (LOS) angle to the target, ensuring that the target remains within the onboard camera’s field of view during the flight. A simulation flight system is developed and used for neural network training and validation. Experimental results indicate that the quadrotor maintains stable chasing performance through various maneuvers of the fixed-wing UAV and can sustain formation over long durations. Our research explores the use of end-to-end neural networks for UAV formation flying, spanning from perception to control. Full article

Most literature has extensively discussed reinforcement learning (RL) for controlling rotorcraft drones during flight for traversal tasks. However, most studies lack adequate details regarding the design of reward and punishment mechanisms, and there is a limited exploration of the feasibility of applying reinforcement learning in actual flight control following simulation experiments. Consequently, this study focuses on the exploration of reward and punishment design and state input for RL. The simulation environment is constructed using AirSim and Unreal Engine, with onboard camera footage serving as the state input for reinforcement learning. The research investigates three RL algorithms suitable for discrete action training. The Deep Q Network (DQN), Advantage Actor–Critic (A2C), and Proximal Policy Optimization (PPO) were combined with three different reward and punishment design mechanisms for training and testing. The results indicate that employing the PPO algorithm along with a continuous return method as the reward mechanism allows for effective convergence during the training process, achieving a target traversal rate of 71% in the testing environment. Furthermore, this study proposes integrating the YOLOv7-tiny object detection (OD) system to assess the applicability of reinforcement learning in real-world settings. Unifying the state inputs of simulated and OD environments and replacing the original simulated image inputs with a maximum dual-target approach, the experimental simulation achieved a target traversal rate of 52% ultimately. In summary, this research formulates a set of logical frameworks for an RL reward and punishment design deployed with real-time Yolo’s OD implementation synergized as a useful aid for related RL studies. Full article

Recent progress in hand gesture recognition has introduced several natural and intuitive approaches to drone control. However, effectively maneuvering drones in complex environments remains challenging. Drone movements are governed by four independent factors: roll, yaw, pitch, and throttle. Each factor includes three distinct behaviors—increase, decrease, and neutral—necessitating hand gesture vocabularies capable of expressing at least 81 combinations for comprehensive drone control in diverse scenarios. In this paper, we introduce a new set of hand gestures for precise drone control, leveraging an RGB camera sensor. These gestures are categorized into motion-based and posture-based types for efficient management. Then, we develop a lightweight hand gesture recognition algorithm capable of real-time operation on even edge devices, ensuring accurate and timely recognition. Subsequently, we integrate hand gesture recognition into a drone simulator to execute 81 commands for drone flight. Overall, the proposed hand gestures and recognition system offer natural control for complex drone maneuvers. Full article

Positioning and autonomous landing are key technologies for implementing autonomous flight missions across various fields in unmanned aerial vehicle (UAV) systems. This research proposes a visual positioning method based on mirrored field-of-view expansion, providing a visual-based autonomous landing strategy for quadrotor micro-UAVs (MAVs). The forward-facing camera of the MAV obtains a top view through a view transformation lens while retaining the original forward view. Subsequently, the MAV camera captures the ground landing markers in real-time, and the pose of the MAV camera relative to the landing marker is obtained through a virtual-real image conversion technique and the R-PnP pose estimation algorithm. Then, using a camera-IMU external parameter calibration method, the pose transformation relationship between the UAV camera and the MAV body IMU is determined, thereby obtaining the position of the landing marker’s center point relative to the MAV’s body coordinate system. Finally, the ground station sends guidance commands to the UAV based on the position information to execute the autonomous landing task. The indoor and outdoor landing experiments with the DJI Tello MAV demonstrate that the proposed forward-facing camera mirrored field-of-view expansion method and landing marker detection and guidance algorithm successfully enable autonomous landing with an average accuracy of 0.06 m. The results show that this strategy meets the high-precision landing requirements of MAVs. Full article

(1) Background: Air rudders are used to control the flight attitude of aircraft, and their surface quality directly affects flight accuracy and safety. (2) Method: Traditional positioning methods can only obtain defect location information at the image level but cannot determine the defect’s physical surface position on the air rudder, which lacks guidance for subsequent defect repair. We propose a defect physical surface positioning method based on a camera mapping model. (3) Results: Repeated positioning experiments were conducted on three typical surface defects of the air rudder, with a maximum absolute error of 0.53 mm and a maximum uncertainty of 0.26 mm. Through hardware systems and software development, the real-time positioning function for surface defects on the air rudder was realized, with the maximum axial positioning error for real-time defect positioning being 0.38 mm. (4) Conclusions: The proposed defect positioning method meets the required accuracy, providing a basis for surface defect repair in the air rudder manufacturing process. It also offers a new approach for surface defect positioning in similar products, with engineering application value. Full article

Time-of-Flight (ToF) cameras are subject to high levels of noise and errors due to Multi-Path-Interference (MPI). To correct these errors, algorithms and neuronal networks require training data. However, the limited availability of real data has led to the use of physically simulated data, which often involves simplifications and computational constraints. The simulation of such sensors is an essential building block for hardware design and application development. Therefore, the simulation data must capture the major sensor characteristics. This work presents a learning-based approach that leverages high-quality laser scan data to generate realistic ToF camera data. The proposed method employs MCW-Net (Multi-Level Connection and Wide Regional Non-Local Block Network) for domain transfer, transforming laser scan data into the ToF camera domain. Different training variations are explored using a real-world dataset. Additionally, a noise model is introduced to compensate for the lack of noise in the initial step. The effectiveness of the method is evaluated on reference scenes to quantitatively compare to physically simulated data. Full article

Accurate and timely estimation of oat biomass is crucial for the development of sustainable and efficient agricultural practices. This research focused on estimating and predicting forage oat biomass using UAV and agronomic variables. A Matrice 300 equipped with a multispectral camera was used for 14 flights, capturing 21 spectral indices per flight. Concurrently, agronomic data were collected at six stages synchronized with UAV flights. Data analysis involved correlations and Principal Component Analysis (PCA) to identify significant variables. Predictive models for forage biomass were developed using various machine learning techniques: linear regression, Random Forests (RFs), Support Vector Machines (SVMs), and Neural Networks (NNs). The Random Forest model showed the best performance, with a coefficient of determination R² of 0.52 on the test set, followed by Support Vector Machines with an R² of 0.50. Differences in root mean square error (RMSE) and mean absolute error (MAE) among the models highlighted variations in prediction accuracy. This study underscores the effectiveness of photogrammetry, UAV, and machine learning in estimating forage biomass, demonstrating that the proposed approach can provide relatively accurate estimations for this purpose. Full article

This study investigates the possibility of utilising a drone equipped with a thermal camera to monitor the spatial distribution of red deer (Cervus elaphus) and to determine their behavioural patterns, as well as preferences for vegetation types in a moor in Denmark. The spatial distribution of red deer was mapped according to time of day and vegetation types. Reed deer were separated manually from fallow deer (Dama dama) due to varying footage quality. Automated object detection from thermal camera footage was used to identification of two behaviours, “Eating” and “Lying”, enabling insights into the behavioural patterns of red deer in different vegetation types. The results showed a migration of red deer from the moors to agricultural fields during the night. The higher proportion of time spent eating in agricultural grass fields compared to two natural vegetation types, “Grey dune” and “Decalcified fixed dune”, indicates that fields are important foraging habitats for red deer. The red deer populations were observed significantly later on grass fields compared to the natural vegetation types. This may be due to human disturbance or lack of randomisation of the flight time with the drone. Further studies are suggested across different seasons as well as the time of day for a better understanding of the annual and diurnal foraging patterns of red deer. Full article

Drones have been attracting significant attention in the field of agriculture. They can be used for various tasks such as spraying pesticides, monitoring pests, and assessing crop growth. Sensors are also widely used in agriculture to monitor environmental parameters such as soil moisture and temperature. Due to the high cost of communication infrastructure and radio-wave modules, the adoption of high-density sensing systems in agriculture is limited. To address this issue, we propose an agricultural sensor network system using drones and Optical Camera Communication (OCC). The idea is to transmit sensor data from LED panels mounted on sensor nodes and receive the data using a drone-mounted camera. This enables high-density sensing at low cost and can be deployed in areas with underdeveloped infrastructure and radio silence. We propose a trajectory control algorithm for the receiving drone to efficiently collect the sensor data. From computer simulations, we confirmed that the proposed algorithm reduces total flight time by $30\%$ compared to a shortest-path algorithm. We also conducted a preliminary experiment at a leaf mustard farm in Kamitonda-cho, Wakayama, Japan, to demonstrate the effectiveness of the proposed system. We collected 5178 images of LED panels with a drone-mounted camera to train YOLOv5 for object detection. With simple On–Off Keying (OOK) modulation, we achieved sufficiently low bit error rates (BERs) under ${10}^{-3}$ in the real-world environment. The experimental results show that the proposed system is applicable for drone-based sensor data collection in agriculture. Full article

The mining industry faces significant challenges in production costs, environmental protection, and worker safety, necessitating the development of autonomous systems. This study presents the design and implementation of a robust rock centroid localization system for mining robotic applications, particularly rock-breaking hammers. The system comprises three phases: assembly, data acquisition, and data processing. Environmental sensing was accomplished using a Basler Blaze 101 three-dimensional (3D) Time-of-Flight (ToF) camera. The data processing phase incorporated advanced algorithms, including Bird’s-Eye View (BEV) image conversion and You Only Look Once (YOLO) v8x-Seg instance segmentation. The system’s performance was evaluated using a comprehensive dataset of 627 point clouds, including samples from real mining environments. The system achieved efficient processing times of approximately 5 s. Segmentation accuracy was evaluated using the Intersection over Union (IoU), reaching 95.10%. Localization precision was measured by the Euclidean distance in the XY plane (ED_XY), achieving 0.0128 m. The normalized error (e_norm) on the X and Y axes did not exceed 2.3%. Additionally, the system demonstrated high reliability with R² values close to 1 for the X and Y axes, and maintained performance under various lighting conditions and in the presence of suspended particles. The Mean Absolute Error (MAE) in the Z axis was 0.0333 m, addressing challenges in depth estimation. A sensitivity analysis was conducted to assess the model’s robustness, revealing consistent performance across brightness and contrast variations, with an IoU ranging from 92.88% to 96.10%, while showing greater sensitivity to rotations. Full article

With the continuous development of science and technology, laser ranging technology will become more efficient, convenient, and widespread, and it has been widely used in the fields of medicine, engineering, video games, and three-dimensional imaging. A time-of-flight (ToF) camera is a three-dimensional stereo imaging device with the advantages of small size, small measurement error, and strong anti-interference ability. However, compared to traditional sensors, ToF cameras typically exhibit lower resolution and signal-to-noise ratio due to inevitable noise from multipath interference and mixed pixels during usage. Additionally, in environments with scattering media, the information about objects gets scattered multiple times, making it challenging for ToF cameras to obtain effective object information. To address these issues, we propose a solution that combines ToF cameras with single-pixel imaging theory. Leveraging intensity information acquired by ToF cameras, we apply various reconstruction algorithms to reconstruct the object’s image. Under undersampling conditions, our reconstruction approach yields higher peak signal-to-noise ratio compared to the raw camera image, significantly improving the quality of the target object’s image. Furthermore, when ToF cameras fail in environments with scattering media, our proposed approach successfully reconstructs the object’s image when the camera is imaging through the scattering medium. This experimental demonstration effectively reduces the noise and direct ambient light generated by the ToF camera itself, while opening up the potential application of ToF cameras in challenging environments, such as scattering media or underwater. Full article

This article investigates the potential for using low-cost depth cameras to estimate the maturity of green table grapes after they have been harvested. Time-of-flight (Kinect Azure) and LiDAR (Intel L515) depth cameras were used to capture depth scans of green table grape berries over time. The depth scans of the grapes are distorted due to the diffused scattering of the light emitted from the cameras within the berries. This causes a distance bias where a grape berry appears to be further from the camera than it is. As the grape aged, the shape of the peak corresponding to the grape became increasingly flattened in shape, resulting in an increased distance bias over time. The distance bias variation with time was able to be fitted with an $R^2$ value of 0.969 for the Kinect Azure and an average of 0.904 for the Intel L515. This work shows that there is potential to use time-of-flight and LIDAR cameras for estimating grape maturity postharvest in a non-contact and nondestructive manner. Full article

With the rapid development of the shipping industry, the number of ships is continuously increasing, and maritime accidents happen frequently. In recent years, computer vision and drone flight control technology have continuously developed, making drones widely used in related fields such as maritime target detection. Compared to the cameras fixed on ships, a greater flexibility and a wider field of view is provided by cameras equipped on drones. However, there are still some challenges in high-altitude detection with drones. Firstly, from a top-down view, the shapes of ships are very different from ordinary views. Secondly, it is difficult to achieve faster detection speeds because of limited computing resources. To solve these problems, we propose YOLOv7-DyGSConv, a deep learning-based model for detecting ships in real-time videos captured by drones. The model is built on YOLOv7 with an attention mechanism, which enhances the ability to capture targets. Furthermore, the Conv in the Neck of the YOLOv7 model is replaced with the GSConv, which reduces the complexity of the model and improves the detection speed and detection accuracy. In addition, to compensate for the scarcity of ship datasets in top-down views, a ship detection dataset containing 2842 images taken by drones or with a top-down view is constructed in the research. We conducted experiments on our dataset, and the results showed that the proposed model reduced the parameters by 16.2%, the detection accuracy increased by 3.4%, and the detection speed increased by 13.3% compared with YOLOv7. Full article