Open AccessArticle

Research on Shoveling Position Analysis and Recognition of Unmanned Loaders for Gravel Piles

Hanwen Zhang

^1,2,

Sun Jin

^1,2,*,

Bing Li

^1,2,*,

Bo Xu

^1,2,

Yuanbin Xiao

^1,2 and

Weixin Zhou

^1,2

School of Mechanical and Automotive Engineering, Guangxi University of Science and Technology, Liuzhou 545006, China

Guangxi Collaborative Innovation Centre for Earthmoving Machinery, Guangxi University of Science and Technology, Liuzhou 545006, China

Authors to whom correspondence should be addressed.

Appl. Sci. 2024, 14(23), 11036; https://doi.org/10.3390/app142311036

Submission received: 21 October 2024 / Revised: 20 November 2024 / Accepted: 24 November 2024 / Published: 27 November 2024

Download

Browse Figures

Figure 1
Overall research methodology framework. "> Figure 1 Cont.
Overall research methodology framework. "> Figure 2
Loader execution device model, among them: 1. Bucket 2. Linkage 3. Boom 4. Rocker 5. Bucket hydraulic rod 6. Bucket hydraulic cylinder 7. Frame 8. Boom hydraulic cylinder 9. Boom hydraulic rod. "> Figure 3
Loader bucket tip trajectory. "> Figure 4
Crushed stone particle model. "> Figure 5
Modeling convex and concave areas of shoveling gravel in a coupled EDEM and Adams simulation, among them: (a) is the simulation of shoveling the convex part of gravel; (b) simulation of shoveling concave part of gravel. "> Figure 6
The lateral force of crushed stone on the bucket. "> Figure 7
The gravel pile after the shovel is finished. "> Figure 8
Generation of rubble piles with different drop geometries, among them: (a) cylindrical geometry; (b) rectangular geometry. "> Figure 9
Resistance curve during the insertion stage. "> Figure 10
Resistance curve during the lifting stage. "> Figure 11
Comparison of edge curvature for simulation I. The three lines represent the different edges formed by the generation of 3375 kg of rubble. H represents the fitted edge curvature. "> Figure 12
Comparison of edge curvature for simulation II. The three lines represent the different edges formed by the generation of 5063 kg of rubble. H represents the fitted edge curvature. "> Figure 13
Comparison of edge curvature for simulation III. The three lines represent the different edges formed by the generation of 6750 kg of rubble. H represents the fitted edge curvature. "> Figure 14
Radar chart area corresponding to different edge curvatures of 3375 kg stockpile. "> Figure 15
Radar chart area corresponding to different edge curvatures of 5063 kg stockpile. "> Figure 16
Radar chart area corresponding to different edge curvatures of 6750 kg stockpile. "> Figure 17
Data augmentation, among them: (a) add Gaussian noise to the original image; (b) add overexposure to the original image; (c) add fog noise to the original image; (d) the original image is rotated and tilted by <math display="inline"><semantics> <mrow> <mo>±</mo> <msup> <mn>5</mn> <mo>∘</mo> </msup> </mrow> </semantics></math> to present the uneven working environment. "> Figure 18
Network structure diagram, where: (a) is the original YOLOv5s network structure diagram. (b) is the improved YOLOv5s network structure diagram in this paper. "> Figure 19
(a) Standard convolution module; (b) ghost convolution module. "> Figure 20
The bottleneck structure of ghost module, where: (a) step size is 1; (b) step size is 2. "> Figure 21
RFB module. "> Figure 22
Structure of CBAM. "> Figure 23
C3-CBAM structure diagram, where: (a) C3-CBAM structure diagram; (b) structure diagram of CBAM bottleneck in C3. "> Figure 24
Loss function curve of the improved model. "> Figure 25
The performance advantage of the improved YOLOv5s model. (a) The change in mAP@0.5 performance indicators of different model algorithms on the data set. (b) The classification performance of the improved YOLOv5s model is evaluated based on the P–R curve. In the figure, the closer the curve is to the upper right corner, the better the model performance. "> Figure 26
Test bench for detection device. "> Figure 27
Experimental results: (a) The image labels of this column of images used for training. (b) This column is the detection result of the original YOLOv5s model. (c) This column shows the detection results of the improved YOLOv5s model. ">

Versions Notes

Abstract

Gravel is the most frequently used material in infrastructure construction. However, the irregular shape of the gravel pile makes it challenging for the loader to predict a stable shoveling position, which can easily result in partial collapse or a complete landslide, thereby posing a serious threat to the equipment. In view of the imperfect method of determining the shoveling position of the pile by the current unmanned loader and the high hardware requirements for the deployment of the identification model, this paper first establishes a mathematical model of the loader, and preliminarily determines the influence of the concave and convex edges of the gravel pile on the shoveling position selection through discrete element joint simulation; secondly, the influence of the pile with different edge curvatures on the loader operation process is analyzed in the simulation software, and the radar map is used to further identify the superior position features; finally, Ghost Net is used as the backbone network, the RFB module is introduced into the Backbone, and the CBAM attention mechanism is integrated into the C3 module to identify the lightweight YOLOv5s shoveling position. Discrete element analysis and a lightweight network model were used in the above study to find the safest and most effective shoveling positions. During the test that mimicked how the loader would actually shovel, the number of parameters in the improved model was cut down to 32.5% of the original, the number of calculations was cut down to about 55.2% of the original, and the average accuracy of finding the shoveling position of the gravel pile reached 98%.

Keywords:

gravel pile; unmanned loader; discrete element co-simulation; shovel position; lightweight network model

1. Introduction

Loaders are widely used in construction sites, mining sites, and other places, and they are important tools in the field of modern construction and engineering. With the rise of the digital revolution, the construction machinery industry is accelerating its transformation to intelligence [1]. However, at present, loaders mainly rely on manual operation for shoveling operations, and there are many problems in the entire shoveling process: First, the operating environment of the loader is relatively complex, potentially posing a threat to the driver’s personal safety. Secondly, the shoveling operation process heavily relies on the driver’s experience and lacks unified standards, which can significantly impact the efficiency and quality of the project [2]. Realizing unmanned operation of loaders has become an important research topic in order to improve the operating efficiency of these machines.

Currently, both XCMG and Doosan Corporation in South Korea have successfully implemented unmanned loader operations. However, during the operation, the loader can only repeatedly operate at the same position and cannot independently determine the shoveling position. Shoving at the same position blindly will inevitably create a large hole there, or excessive shoveling resistance will lift the rear wheel off the ground. It can be seen that correctly judging the shoveling position has an enormous impact. The correct shoveling position not only ensures that the loader collects and fills materials effectively, but also ensures the safety of loading operations and prevents accidents. Therefore, correctly identifying the shoveling position is crucial for unmanned loaders to reduce safety accidents and improve operating efficiency.

Currently, extensive research is underway in the field of autonomous operation of unmanned loaders, specifically focusing on the material shoveling process. To achieve autonomous material shoveling, unmanned loaders must first obtain the location information and shape characteristics of the material pile in real time. Therefore, identifying the material pile and reconstructing its morphological characteristics are crucial for selecting the appropriate shoveling material position. In the field of 3D reconstruction of material piles, the literature [3] discretized the map and adopted a grid-based method. An algorithm used the relative angle between the two columnar models on the target grid and the adjacent grid, along with the height information in each grid, to estimate the shape of the material pile, leading to the determination of multiple shoveling positions. However, discretizing the map into a grid will inevitably result in information loss, as the actual material shape is a continuous curve and surface, and the grid method only provides an approximate representation. Given the limitations of this approximate method, collecting point cloud data of the material pile using LiDAR is a viable option for achieving 3D reconstruction. Reference [4] has installed multiple LiDAR in the working environment to achieve distance measurement and 3D reconstruction of the pile. This solution allows the loader to obtain the shape changes in the pile in real time during the shoveling operation. However, this solution has certain usage restrictions. The majority of loaders’ actual application scenarios are unstructured, and their fixed working locations limit the loader’s working environment range. Given the common usage scenarios of loaders, the literature [5] recommends installing a laser radar on the loader. This allows the loader to obtain point clouds by scanning the surface of the pile during the operation phase. The loader can then use the fitted quadratic surface to estimate the local convex area and distance features that are suitable for the shoveling position. This solution not only meets the real-time requirements of the operation process but also estimates the local convex area by fitting the quadratic surface using the laser radar, enabling the loader to dynamically adjust during the shoveling process. It is undeniable that this solution improves the automation and intelligence level of the loader operation. However, during actual operation, dust will inevitably appear in the environment, hindering the laser radar’s ability to penetrate effectively and resulting in information loss. Since a single sensor cannot effectively solve the problem, the literature [6] adopted a multi-sensor fusion method of LiDAR and vision. This method uses visual sensors to locate the pile, while LiDAR performs three-dimensional reconstruction of the pile surface. This method enhances the flexibility and real-time performance of the loader’s shoveling process. However, while this brings about efficient operation, it also increases equipment costs, and the loader can only shovel one position at a time during the shoveling operation. When dealing with a pile of materials that has an irregular surface height and uneven ground contact edges, the loader may find multiple positions that meet the required shoveling conditions. At this time, how the loader should determine the appropriate shoveling position has an important impact on the overall operation’s efficiency and safety. Among these challenges, the development of visual detection technology has presented a novel solution. Due to its high image resolution and rich pixel information, it can solve the problem of detecting targets in complex environments. Therefore, the engineering field has widely used visual target detection based on deep learning [7]. For example, reference [8] used the YOLOv5 network to solve the problem that low light, low contrast, and objects with similar crack features in tunnel environments lead to less feature information and thus affect the crack recognition effect; reference [9] used deep learning methods to solve the problem that complex scenes and overlapping vehicles in mining environments affect tracking detection, recognition, and counting. In reference [10], YOLOv5s were employed to investigate the identification of bucket working angles, with the aim of reducing energy consumption during loader shoveling operations. The loader bucket posture can adjust in time to achieve energy savings when material resistance causes the angle to change. Reference [11] used a neural network to identify the change curve of the engine power when the loader shovels different materials, thereby controlling the extension of the loader boom cylinder and the bucket angle, thereby achieving intelligent energy saving. In reference [12], a YOLOv5-based method processed multi-sensor data to achieve obstacle recognition, enabling the excavator to identify and detect objects in a dark environment at night and ensuring safe operation.

This paper proposes a method that combines position feature analysis based on edge curvature with an improved deep learning network lightweight recognition model. We select gravel particles as the operation object for two reasons. On the one hand, in engineering operations, gravel is an important raw material for many infrastructure constructions, and it is common for loaders to shovel gravel piles; on the other hand, the irregular shape of gravel makes it difficult for unmanned loaders to find a stable force position when shoveling, which makes it complicated to determine the characteristics and recognition of the shoveling position. This solution employs the coupled simulation of EDEM (engineering discrete element method) and Adams (automatic dynamic analysis of mechanical systems) to initially determine the shape of the shoveling area. It then uses the coupled simulation to find out how the different curves of the gravel pile’s edge affect the choice of the shoveling area. In this process, we not only consider the influence of operating resistance but also introduce the quality of the shoveled material as a comprehensive evaluation factor to identify the characteristic elements that can identify the shoveling location. We then use the improved lightweight deep learning network model to identify the location that possesses these characteristics.

The contributions of this paper are as follows:

Construct a gravel pile model in EDEM and establish a kinematic model of the loader actuator in Adams. Conduct a dynamic simulation using a specific coupling interface to initially identify the features of the pile’s edge shape that can reduce the lateral force on the bucket. This part corresponds to Section 3 of the article.
Further analyze the edge shape features of the initially acquired pile. Similarly, we establish characteristic piles with different edge curvatures in EDEM and couple them with the kinematic model in Adams for simulation, aiming to determine the operating resistance of the crushed stone material on the bucket and the shoveling quality. We use the obtained resistance and shoveling quality as comprehensive evaluation factors to identify the optimal feature among multiple identical features and set it as the target for deep learning model recognition. This content corresponds to Section 4 of the article.
The current recognition model has high hardware requirements and is not simple to deploy on mobile devices such as unmanned loaders. Therefore, we have optimized YOLOv5s with lightweight improvements to meet the needs of shovel position detection. The improvement here is from the perspective of model optimization, focusing on improving the model’s own capabilities; it does not involve considerations related to actual deployment.

2. Overall Framework

This paper proposes a method that combines position detection features based on edge curvature with an improved lightweight model of a deep learning network. The overall framework is shown in Figure 1, which is divided into five modules in the order of the overall method:

(a): We constructed a scene in a laboratory setting based on the typical gravel accumulation situation, and we took numerous images of various accumulation states.
(b): The mathematical model of the loader and the gravel particle model were established by using the EDEM and ADAMS coupling simulation method, and the influence of the concave and convex features on the location selection was analyzed from a two-dimensional perspective.
(c): Following the initial determination of the shoveling position’s characteristic shape, the simulation software primarily establishes material piles with varying edge curvatures to identify distinct areas. Through coupled simulation, the influence of the force condition and shoveling quality during the shoveling operation is analyzed, respectively, and then a comprehensive evaluation method is used to reflect the advantages and disadvantages of the shoveling conditions at different positions.
(d): We obtain results suitable for identifying the characteristic position of the scooping operation by combining the simulation analysis conclusions of modules (b) and (c). Simultaneously, we employ a series of lightweight improvement methods to simplify the deployment of the network model on mobile devices like loaders, thereby reducing the number of model parameters and calculation time while maintaining detection accuracy.
(e): We successfully identify the optimal scooping operation position for the crushed stone pile.

3. Joint Simulation Analysis to Preliminarily Determine Location Features

3.1. Build Loader Actuator Model

1.2 t loaders are widely used in small projects and specific operation scenarios, and the purchase, operation, and maintenance costs are relatively low. Many companies have models of this tonnage. Therefore, this article takes the 1.2 t hydraulic loader as the execution object. Since the loader execution equipment contains many parts, and these parts have little effect on its operation and calculation results, the loader’s working device can be simplified while ensuring that the movement process and calculation results are not affected. The following is a 3D model of the loader execution device established using SolidWorks 2021 software, as shown in Figure 2.

The actual shoveling operation of the loader is mainly divided into three stages: bucket insertion stage, bucket lifting stage, and bucket retraction stage. Since this paper aims to identify the optimal shoveling position, it is necessary to analyze the resistance encountered during the bucket insertion stage and the lifting stage. The operation process of completing these two stages is mainly achieved through the expansion and contraction of the bucket cylinder and the boom cylinder.

3.2. Actuator Multi-Body Dynamics Establishment

Adams multibody dynamics simulation software is frequently used in engineering to develop and analyze complicated mechanical systems. Three crucial considerations to consider while utilizing Adams for loader model analysis:

(1): Kinematic Constraints

Kinematic constraints describe the restrictions between the various parts of the loader, including the steering system, lift arm, and bucket linkage, as well as their range of motion and interrelationships, as shown in Table 1. If redundant constraints are established, the simulation program may arbitrarily eliminate some constraints, which can lead to errors during motion simulation. Therefore, it is necessary to accurately describe the kinematic constraints between the various parts to ensure that the simulation is consistent with actual conditions.

(2): Model Accuracy and Precision

This mainly refers to the parameters of the geometry, mass distribution, and inertia characteristics of the loader model accurately reflecting the actual situation to ensure the accuracy and reliability of the simulation results. The material of the working device is set to steel in Adams, with a density of 7801 kg/m³, Young’s modulus of 2.07 × 1011 N/m², and Poisson’s ratio of 0.29.

(3): Load the excavation trajectory

Loaders can work in a variety of ways, but their efficiency is often determined by the worker’s experience. Common activities include single and multiple filling, excavating, and compound filling. Among them, the compound shovel excavation method considers the flexibility of multiple fillings in its operation and can adapt to varied loading conditions while boosting loading efficiency. As a result, this paper selects the compound filling trajectory for simulation. In Adams, the driving mechanism of the frame movement and the configuration of each hydraulic cylinder are shown in Table 2, and Figure 3 depicts the trajectory curve of the loader bucket tip.

3.3. Discrete Element Modeling and Coupled Simulation of Bulk Materials

EDEM has demonstrated significant effectiveness in the field of research on material loading by loaders. First, EDEM can accurately simulate the physical properties of materials. For all types of materials handled by loaders, whether they are loose gravel or other materials with different particle sizes, densities, and friction coefficients, EDEM can accurately model them through its powerful parameter setting function. This accurate material model construction makes the simulation of the interaction between the loader bucket and the material more realistic. Second, EDEM is highly effective in simulating the kinematics and dynamics of the loader. It can accurately reproduce the various actions of the loader during loading, including bucket insertion, lifting, and turning. By setting the motion parameters of each joint and component of the loader, EDEM can calculate the forces acting on the loader during different loading stages, which provides reliable data support for studying the selection of the loader’s loading position.

If the particles are filled according to the shape of real particles, the results will be more accurate, but the simulation time cost will increase and a large amount of data will be generated, which is not conducive to subsequent post-processing and result analysis. Therefore, according to the outer contour of the actual gravel particles, spherical particles are used in the EDEM 2023 software to fill, and the size and shape are adjusted. The resulting gravel particle model is shown in Figure 4. Using spherical particles to represent gravel has advantages in computational efficiency and practicality and is a simplified method. Even though this simplified method does not fully capture the microscopic limitations caused by the angular properties of real gravel, we are more interested in the overall stability when figuring out where the unmanned loader should shovel from a macro perspective, so the inaccurate microscopic behavior caused by the simplification of spherical particles does not have a big effect on the simulation results.

Given the relatively stable material properties, they are not affected by external variables. The contact parameters for materials primarily consist of the static friction coefficient, rolling friction coefficient, and collision restitution coefficient, which are assessed by sliding plate and drop tests [13]. The material property characteristics and contact property parameters for crushed stone and steel are presented in Table 3 and Table 4, respectively.

The above process established a pile of crushed stone particles in EDEM and determined the motion trajectory of the loader model in Adams. A specific coupling interface enables bidirectional data transmission between the two. During the coupling simulation process, EDEM calculated the force of particles on the mechanical structure and transmitted it to Adams. Adams calculated the motion state of the mechanical structure and fed it back to EDEM. This coupling simulation can fully and realistically simulate the complex working conditions involving the interaction between granular materials and mechanical systems. EDEM-Adams coupling simulation conducts comprehensive functional verification and optimization to reduce the cost and time of actual prototype development.

When the loader is shoveling, especially when facing an uneven material surface, there may be increased resistance, tire slippage, and engine speed reduction. According to the “Safety Technical Operation Regulations for Loaders”, shoveling operations should be stopped when encountering these situations. A forced operation could lead to the loader losing its balance and toppling over. Therefore, it is necessary to analyze the lateral force of the loader bucket based on the surface morphology of the pile to ensure the loader is produced safely during the shoveling process. The loader’s shoveling process can be divided into two key stages: bucket horizontal insertion and bucket lifting [14]. In these two stages, the force on the side of the loader bucket is crucial. Relevant verification was carried out through EDEM simulation, as shown in Figure 5.

The force acting on the side of the bucket is extracted by the EDEM post-processing, as shown in Figure 6, where 0–3 s is the scooping phase, 3–6 s is the lifting phase, and after 6 s the bucket retracts. It can be seen from Figure 6 that during the insertion phase, the position of the convex part exerts a much lower lateral force on the bucket’s than the concave part. Therefore, the convex area of the bucket can effectively prevent the bucket from being subjected to excessive lateral forces during operation, ensuring safety during the working process and laying the foundation for subsequent evaluation of the bucket’s contact point position.

4. Evaluation and Establishment of Different Location Features Based on Joint Simulation

4.1. Model Establishment and Extraction of Gravel Piles with Different Edges

After initially determining the characteristic shape of the shoveling position, it was found that the material accumulation shape after the loader shoveled changed, and the bucket shovel area showed a concave shape in the middle and a convex shape on both sides, as shown in Figure 7. From this, it can be judged that the next shoveling position point should be selected in the convex area, such as the areas represented by the three different color boxes in Figure 7. Therefore, distinguishing different convex areas is an important link to ensuring energy savings and high efficiency for the loader in production.

This paper adopts image recognition technology and uses the image features of different convex areas as the basis for identification. As can be seen in Figure 7, there are differences in the curvature of the edges of the convex areas. Man et al. [15] used the discrete element method (DEM) to play around with different kinds of initial column cross-sections and see how the granular columns broke apart. We borrowed their method to verify the influence of different edge curvatures on the shoveling process.

In this paper, a cylindrical barrel with a cross-sectional area of 2.54 m² and a rectangular barrel are built, and they are filled with gravel particles with a mass of 3375 kg, 5063 kg, and 6750 kg, respectively. The particle generation process is shown in Figure 8. First, the pile with different edge heights is obtained by lifting the barrel column. Second, the edge shape is obtained by binarizing the stacking image from a top-down view in MATLAB. Then, the pixel coordinate points of the edge contour curve are extracted using the Roberts edge detection method [16].

The Roberts operator is effective for detecting rugged edges of piles and handling low-noise images. It estimates potential edges in the image by calculating the difference between adjacent pixels in diagonal directions. For example, the following operator templates are given, and the gradient magnitudes

g_{x}

and

g_{y}

at pixel point P2 are calculated as follows: Applsci 14 11036 i001

g_{x} = \frac{\partial f}{\partial x} = P 6 - P 2

(1)

g_{y} = \frac{\partial f}{\partial y} = P 5 - P 3

(2)

Finally, the curve is fitted, and the average curvature value is calculated. The formula is shown in (4). Assume that the equation for the curve is shown in (3). Using the parametric equation derivation method, the curvature matrix can be divided by the number of points to obtain the average curvature

H

, where

N

is the number of points on the approximate curve.

\{\begin{matrix} x = φ (t) \\ y = ω (t) \end{matrix}

(3)

H = \frac{|φ^{'} (t) ω^{''} (t) - ω^{'} (t) φ^{''} (t)|}{{[φ^{' 2} (t) + ω^{' 2} (t)]}^{\frac{3}{2}} \times N}

(4)

4.2. Coupling Simulation and Establishment of Comprehensive Evaluation of Shovel Position

When post-processing the EDEM simulation of shoveling a pile of gravel, the resistance curves for the insertion stage and the lifting stage can be obtained, as shown in Figure 9 and Figure 10, respectively.

Friction and particle cutting force cause resistance during the insertion stage when the bucket and gravel interact. The resistance in the lifting stage is affected by the viscosity between the particles in the bucket. Analyzing these two stages separately can provide a clearer understanding of the resistance characteristics of different stages.

Since a single evaluation method cannot fully and accurately reflect the resistance characteristics at different stages and under different influencing factors, this paper adopts a comprehensive evaluation method to indirectly reflect the situation of the loader shoveling the material pile [17]. This is done to express the influence of different convex feature shoveling positions on the loader and then determine the shape features that are better for the loader shoveling situation.

After EDEM simulated the loader shoveling operation, the three most important factors were the maximum pressure in each direction of the bucket insertion and lifting stages, as well as the mass of the material shoveled by the bucket. These were then used to fully evaluate the pros and cons of each shoveling position. Among them, the maximum insertion resistance is a key indicator to measure the loader’s ability to overcome material resistance during operation. If the resistance is too large, it may put too much burden on the loader’s structure and power system, affecting the service life and reliability of the equipment. The maximum lifting resistance reflects the capacity and stability of the loader. A smaller maximum lifting resistance means that the loader can complete the lifting action faster, improve the operation cycle time, and thus increase the load per unit time. The maximum filling mass is crucial to evaluating the production efficiency and economic benefits of the loader at work. A larger maximum filling mass means that the loader can transport more materials in one operation, reduce the number of operation cycles, and reduce operating costs. Then, the corresponding degree value is calculated using the radar chart evaluation method. For the evaluation of the loader operation process, multiple key indicators are usually involved. Each indicator corresponds to an axis of the radar chart. The scale on the axis can clearly show the specific value and relative size relationship of each indicator, thus providing an intuitive basis for the comprehensive evaluation of the loader operation. Finally, the pros and cons of the shovel position are reflected by comparing the size of the degree value.

Given that the three indicators have different measurement units and scales, which can significantly impact the results of modeling and analysis, it is necessary to normalize the data and map it to a specific range to ensure comparability among the different features. Since Gaussian normalization has a certain resistance to noise in the data, it is more robust. Gaussian normalization can be calculated using the following formula:

X^{*} = \frac{x - μ}{σ}

(5)

where

X^{*}

is the eigenvalue after Gaussian normalization,

x

is the original value of the feature,

μ

is the mean of the feature in all samples, and

σ

is the standard deviation of the feature in all samples.

In order to distinguish the characteristics of different convex areas of the stockpile, the analysis still uses the EDEM-Adams coupling simulation. The overall simulation analysis process involves the following steps: First, EDEM creates piles of crushed stone particles weighing 3375 kg, 5063 kg, and 6750 kg. Then, as shown in Figure 11, Figure 12 and Figure 13, the edge curvatures of these piles are taken out.

Next, we conduct the coupled simulation experiment using EDEM and Adams. After completion, we can extract the data we need in the post-processing part of EDEM, as shown in Table 5. Table 5 clearly shows that in EDEM, three different masses of crushed stone particles generate a total of nine different curvatures of the pile edge in different material drop tubes. The dynamic simulation, in conjunction with Adams, yields the values of the maximum insertion resistance, maximum lifting resistance, and maximum filling mass. The last column in the table is the area size enclosed by the three indicators on the radar chart after normalization.

The radar chart’s area surrounded by the calculation after normalization can be used to measure and contrast the overall performance of various samples or entities in various dimensions. The evaluation outcomes can be clearly shown, as seen in Figure 14, Figure 15 and Figure 16. In order to ensure that the load on the working device during the insertion and lifting stages of the loader shovel is small while ensuring a certain shoveling efficiency, the shoveling quality is taken inversely before normalizing the data, with the aim of making the three evaluation indicators have the same directional size. Therefore, in the visualization results of the radar chart, if the area enclosed by the three evaluation indicators is larger, it indicates that the shoveling effect is not adequate under the edge curvature; otherwise, it is better.

In summary, our simulation comparative analysis revealed that when a loader encounters a pile of materials in various states, if it shovels from the convex area of the pile’s edge with a small curvature, it can effectively reduce the resistance of the shoveling process and ensure a certain shoveling efficiency. This discovery also establishes a foundation for identifying the shoveling position, a crucial step towards achieving real-time and low-energy consumption goals for unmanned loaders.

5. Lightweight Identification Model for Shoveling Position

Through simulation analysis and comprehensive evaluation of the shoveling position, the recognition features for distinguishing its quality were determined. Although the recognition features have been determined, there are still problems with the large number of parameters and high computational complexity of the target detection model in the loader’s real-time cyclic shoveling position recognition. The high hardware equipment requirements make deployment on mobile devices challenging. To make online cyclic shoveling position recognition work for unmanned loaders, it is especially important to make the target detection model of the shoveling position as light as possible while still making sure that the detection accuracy on mobile devices.

5.1. Data Sample Collection

This paper takes gravel as the research object and studies it in a laboratory environment by simulating the process of a loader scooping up a gravel pile. In order to allow the model to access more data from different angles and conditions, and thus learn a wider range of features, we have introduced a variety of transformations and adopted data augmentation methods to increase the richness of the data so that the model can learn more data features in different forms, reduce overfitting, and improve the robustness of the model. This paper uses noise processing, overexposure, fog processing, and rotation, as shown in Figure 17.

Through the above method, we finally obtained 2400 images of material accumulation. Based on these images, we randomly divided them into a training set of 70%, a test set of 20%, and a validation set of 10%. The calculation shows that there are 1680 images in the training set, 480 in the test set, and 240 in the validation set. This division facilitates the subsequent training, testing, and validation of the model, ensuring its performance and generalization ability.

5.2. Network Model and Improvements

YOLOv5 is well regarded in practical industrial applications. The latest version of YOLO has increased the number of layers and parameters in the network architecture, resulting in a larger model size, requiring more computing resources to deploy and operate, and an increasing amount of data for training. Therefore, based on the current operational requirements for unmanned loaders, YOLOv5s was selected. It has fewer parameters and a smaller model size and can be improved to be suitable for mobile devices and scenarios with limited computing resources.

The architecture of YOLOv5s primarily comprises three components: the input, the backbone network, and the detecting head [18]. Cross-shading and residual connections connect the backbone network to other networks. These make it easier for information to flow and for features to be used efficiently while keeping the network’s depth. Furthermore, the detection head employs convolutional and pooling layers to ascertain the target’s location and category information from the feature map derived by the backbone network. The network structure diagram of YOLOv5s is shown in Figure 18a.

To make the recognition model more lightweight, reduce its size, and ensure the loader’s accuracy in identifying shovel point positions, this study employs four optimization strategies based on the original model. The improved YOLOv5s model network is shown in Figure 18b. The main improvements are as follows:

Ghost Net replaces the backbone network of YOLOv5s. Its “ghost module” uses the inherent similarity of feature maps to generate rich feature maps at low cost, reduce the amount of calculation, and achieve a lightweight model, which is convenient for running on mobile devices with limited resources, such as unmanned loaders.
Adding RFB to the Ghost Net backbone can expand the receptive field to obtain broader contextual information, which is crucial for position recognition during loader shoveling operations. Since the loader’s operating environment is complex, accurate position recognition needs to consider the surrounding environment. RFB can assist the model in effectively utilizing environmental information to locate the target, while also fostering multi-scale feature fusion to accommodate the recognition of shoveling targets at varying distances.
We add the C3-CBAM module based on the above-improved model. The C3 module reduces the amount of calculation and enhances the learning ability. The CBAM weights the features based on channel and spatial dimensions. During the loader operation, the channel attention can highlight key feature channels related to the excavation position, while the spatial attention can suppress complex background interference and improve the recognition accuracy.
Changing CIoU to EIoU can make the loss function more accurate and reduce the positioning error of the operating position target, thereby improving the recognition accuracy, which is of vital importance for unmanned loaders to achieve safe production during operation.

5.2.1. The Backbone Network Introduces the Ghost Net Architecture

Ghost Net is a light-weight convolutional network architecture introduced by Huawei’s Noah’s Ark Lab [19]. The core idea of the network is to partition the convolution operation into two components. A limited quantity of ordinary convolutions is executed, succeeded by a block-wise linear convolution on the resultant feature maps to produce numerous supplementary features at minimal expense. The ghost module aims to improve network efficiency by producing additional feature maps from the original feature maps via linear transformation. The execution of the ghost module has three stages: initially, create feature maps utilizing conventional convolution as illustrated in Figure 19a. Secondly, execute a linear transformation on these feature maps to produce redundant feature maps by DW convolution. Third, amalgamate the feature maps produced by convolution with those created by linear transformation, as seen in Figure 19b.

The ghost bottleneck is a bottleneck architecture consisting of ghost modules, with two variations illustrated in Figure 20. The left diagram (a) depicts a backbone with two sequential ghost modules; the initial module augments the channel count, while the subsequent module diminishes it to align with the input channels. This architecture facilitates the extraction of more profound features and enhances the model’s generalization capability. In picture (b) on the right, a depthwise separable convolution with a stride of 2 is added between the two modules in the backbone. This makes the feature maps half as big compared to the input by reducing their height and width. This significantly diminishes the model’s hardware demands and computational intricacy.

5.2.2. Introduction of RFB Module

With the progression of deep learning, efforts are being made to optimize performance, spatial awareness, and computing efficiency to boost the extraction of target feature information and improve outcomes in visual tasks. Presently, some widely utilized techniques for target feature extraction encompass SPP [20], SPPF [21], ASPP [22], SSPCSPC [23], and RFB [24]. SPP fuses features at different levels of an image by pooling feature maps of different scales so that the network can capture characteristics that are important at different scales of the input image. SPPF enhances SPP by incorporating a feature fusion step with multi-scale pooling; hence, it augments the richness and diversity of feature representation. ASPP uses dilated convolutions to improve resolution while keeping the original receptive field. This makes it better at adapting to the edges of objects. The SSPCSPC model acquires spatial information at several scales while employing CSP modules to augment feature variety. RFB strengthens the interaction between features through recursive feature fusion and improves the ability to understand complex scenes. Table 6 displays the pertinent parameters for the aforementioned feature extraction methods.

Table 6 shows that SPP, SPPF, and RFB have low numbers of parameters and computational demands. Nevertheless, ASPP and SSPCSPC have higher numbers of parameters and computational demands, which makes the models more complicated and memory intensive. SPP pooling typically immobilizes and may not adjust to variations in input size, thereby overlooking useful details. SPPF utilizes multi-scale pooling, executing several downsampling operations on the input feature maps, in contrast to SPP. Nonetheless, SPPF generally employs fixed pooling window sizes, which are inflexible and may not facilitate efficient feature extraction in every context. Therefore, it is not the best choice for tasks such as detecting shovel point positions. RFB focuses on capturing the details of local features and, compared to SPPF’s global average pooling, better keeps spatial information and picture details, which makes it easier to find edge features at shovel point positions. The structure is seen in Figure 21.

5.2.3. C3-CBAM Attention Mechanism

The main idea behind the CBAM attention mechanism is to use adaptive weighting of spatial and channel properties in convolutional neural networks. This makes it easier for the networks to quickly pull out the important data from the input [25]. CBAM comprises two primary components: channel attention and spatial attention. Channel attention calculates global statistics for each channel’s activation map, subsequently generating channel weights via normalization, which are then applied to the original feature map to highlight significant channel features. Spatial attention learns about the whole world’s space by using convolution and pooling techniques. It then creates positional weights that show how important different parts of the input image are. Ultimately, these two weights are multiplied element-wise to adjust the spatial dimensions of the original feature map. Figure 22 illustrates the comprehensive structure of CBAM.

The C3-CBAM attention mechanism integrates the CBAM attention module with the C3 module, as seen in Figure 23. The C3 module divides the original feature map into two parts: one part is processed using standard convolution, while the other segment is subjected to multiple residual convolutions. The two components are subsequently integrated using an additional conventional convolution. The C3-CBAM attention mechanism combines both parts, making it easier for the model to understand photos of material pile shoveling positions and to extract more detailed feature representations.

5.2.4. EIoU Loss Function

YOLOv5s uses the CIoU loss function to penalize aspect ratio differences by calculating the diagonal distance of the smallest enclosing box, thereby measuring the similarity of the rectangular boxes [26]. The formula for the CIoU loss function is as follows:

α = \frac{υ}{(1 - I o U) + υ}

(6)

υ = \frac{4 {(\arctan \frac{ω^{g t}}{h^{g t}} - \arctan \frac{ω}{h})}^{2}}{π^{2}}

(7)

L_{C I o U} = 1 - I o U (A, B) + \frac{ρ^{2} (A_{c t r}, B_{c t r})}{c^{2}} + α υ

(8)

Here,

L_{C I o U}

represents the similarity between the predicted box

A

and the ground truth box

B

ρ

denotes the distance between the center points of the predicted box

A

and the ground truth box

B

, with

A_{c t r}

and

B_{c t r}

representing these centers.

c

is the diagonal length of the smallest enclosing box that contains both

A

and

B

ω^{g t}

and

h^{g t}

are the width and height of the ground truth box, while

ω

and h are the width and height of the predicted box.

υ

is a correction factor used to further adjust the loss function.

CIoU can effectively avoid divergence issues during training. However, when the aspect ratios of the predicted box and the ground truth box are the same, the aspect ratio penalty term becomes zero, which clearly leads to a lack of regression optimization for the predicted box. Therefore, replacing the original loss function with the EIoU loss function addresses this issue. EIoU builds upon the penalty term of CIoU by separating the aspect ratio influence factor of the predicted and ground truth boxes, calculating their lengths and widths individually [27]. This approach addresses the issues present in CIoU. The formula for the EIoU loss function is as follows:

L_{E I o U} = 1 - I o U + \frac{ρ^{2} (A_{c t r}, B_{c t r})}{c^{2}} + \frac{ρ^{2} (ω, ω^{g t})}{c_{ω}^{2}} + \frac{ρ^{2} (h, h^{g t})}{c_{h}^{2}}

(9)

Here,

c_{ω}

and

c_{h}

represent the width and height of the smallest enclosing box that contains both boxes.

5.3. Experimental Results and Analysis

The PC utilized in this experiment operates on Windows 11 and is outfitted with an Intel Core i7-9000K CPU, a 6 GB GTX 1060 GPU, and 16 GB of RAM. The development environment utilized is PyCharm, employing Python 3.9 as the programming language. The PyTorch deep learning framework is implemented for training and validating the loader shovel point position recognition model. The training data for the model come from a self-built dataset. The experimental parameter settings are input image dimensions of 640 × 640 pixels, a batch size of 8, 100 training iterations, an initial learning rate of 0.01, and a decay factor of 0.01.

The mean average precision ([email protected]), recall (R), and precision (P) are selected as evaluation indicators for loader shoveling position recognition. Among them, the mean average precision ([email protected]) provides a comprehensive evaluation of the target detection algorithm’s performance across various categories, making it a relatively objective and comprehensive evaluation indicator. A higher [email protected] value means that the algorithm performs well in balancing precision and recall and can detect the target position more accurately while reducing false detections and missed detections. Recall (R) refers to the proportion of correctly identified shoveling positions among all actual correct shoveling positions. The higher the R value, the fewer missed detections can be, thereby saving time and energy and improving work efficiency. Precision (P) refers to the proportion of true shoveling positions among all results detected as shoveling positions. The higher the P value, the lower the false alarm rate of the model. F1 (F1-score) is the harmonic mean of precision (P) and recall (R). Statistics use it as an indicator to measure the accuracy of binary classification models, considering both the accuracy and recall of the classification model. In deep learning classification tasks, the F1 value ranges from 0 to 1, with a maximum value of 1 and a minimum value of 0. A larger F1 value means better model performance. The F1 value is calculated as follows:

F 1 = 2 \times \frac{P \times R}{P + R}

(10)

where P is precision and R is recall.

5.3.1. Comparison of Lightweight Network Models

To make the network model lightweight, we replace the YOLOv5s backbone network with a lightweight network called Ghost Net. Currently, the most widely used mainstream lightweight network structures include MobileNetV3 [28], Shufflenetv2 [29], MobileNeXt [30], etc. We experimentally compare Ghost Net with these three networks. Table 7 displays the results.

Considering the number of parameters, ShuffleNetv2 shows significant advantages. Its number of parameters is only 0.84 M, which is the smallest among the compared models. In stark contrast, Ghost Net and MobileNeXt have a maximum parameter count of 4.42 M. ShuffleNetv2 also exhibits excellent characteristics on the GFlops metric, with a value as low as 1.8. In comparison, the GFlops of MobileNetV3 is 2.3, while MobileNeXt is as high as 8.4, the highest among all models, which means that ShuffleNetv2 has a clear advantage in terms of computational resource consumption. Further observing the performance indicators, we found that Ghost Net performed well in key performance indicators such as recall, precision, F1 score, and [email protected]. In particular, its [email protected] reaches 97.6%, highlighting the excellent detection capabilities of Ghost Net among the compared models. Despite the advantages of ShuffleNetv2 and MobileNetV3 in terms of parameter volume and GFlops, Ghost Net outperforms them in terms of performance. In summary, after weighing multiple key factors, such as model calculation volume, parameter volume, and recognition accuracy, we finally decided to use Ghost Net as the backbone network for the improved model.

5.3.2. Improved YOLOv5s Model Ablation Test Analysis

We introduce the Ghost Net network as the backbone network and integrate the RFB module into it to increase the receptive field and promote multi-scale feature fusion. The neck part of YOLOv5s also incorporates C3-CBAM to suppress complex background interference and enhance recognition accuracy. Simultaneously, we modify the loss function to EIoU, which reduces the shoveling position error of the excavation target and enhances recognition accuracy. We improve the detection accuracy of the lightweight model to make it more suitable for deployment on mobile devices. We conduct ablation experiments on various improvements to the YOLOv5s model. We maintain consistency in the number of iterations, training batch size, and other experiment conditions. Table 8 displays the experimental comparison results.

The ablation experiment comparison shows that on the gravel pile detection dataset, G-v5s (Ghost Net + YOLOv5s) has a lot fewer parameters and GFlops than the original YOLOv5s. However, the F1 value and [email protected] have gone up by 0.2 and 0.7 percentage points, respectively. This shows that the introduction of the combination of Ghost Net and YOLOv5s has produced a synergistic effect. From a classification point of view, the higher F1 value is because Ghost Net reorganized and optimized the features, which makes the features of different types of targets easier to tell apart in the feature space. From a positioning point of view, the model is better at figuring out where the target is within its bounding box. The reason for the improvement in [email protected] is that Ghost Net enhances the model’s ability to understand and utilize target information, thereby improving the detection accuracy of different categories of targets. The F1 value and [email protected] of GB-v5s (Ghost Net + RFB + YOLOv5s) are 95.2% and 97.9%, respectively. When compared to the G-v5s model with the same weight, the [email protected] increases by 0.3 percentage points, while the F1 value decreases by 0.1 percentage points. This shows that the rise in [email protected] is because adding RFB makes the receptive field bigger. This lets the model get more contextual information, which helps it figure out the target’s location and category more accurately, especially when the scene changes a lot. The decrease in the F1 value could potentially be attributed to the model misinterpreting some information that is similar to the target, leading to its decrease. On this basis, based on this, GBC-v5s (Ghost Net + RFB + C3-CBAM + YOLOv5s) shows a slight decrease in GFlops, an increase in F1 value of 0.4 percentage points, and a decrease in [email protected] of 0.1 percentage points compared to GB-v5s. When the F1 value goes up, the C3-CBAM module can better focus on the image’s important features and block out distracting information. This makes it easier for the model to tell the difference between the target and lowers the number of mistakes and missed opportunities. The decrease in [email protected] is due to the fact that while the C3-CBAM module enhances the features, the way it selects or weights them affects the model’s accurate judgment of the target boundary to a certain extent. Therefore, we replace the loss function CIoU with EIoU. EIoU can give the model more accurate feedback by measuring the differences between bounding boxes more accurately. This helps the model make more accurate changes to the prediction box’s size and location during training, which improves the accuracy of its judgment of the target boundary. This makes GBCE-v5s (Ghost Net + RFB + C3-CBAM + EIoU + YOLOv5s) increase its [email protected] by 0.2 percentage points and its F1 value by 0.1 percentage points compared to GBC-v5s. Compared with the original YOLOv5s, GBCE-v5s reduces the number of parameters and GFlops by 2.34 M and 9.1 respectively and increases its F1 value and [email protected] by 0.6 percentage points and 1.1 percentage points, respectively. This shows that the GBCE-v5s model proposed in this paper can ensure the ability of unmanned loaders to effectively detect and identify target locations while meeting the lightweight requirements.

5.3.3. Comparison of Different Object Detection Models

To fully and accurately test how well the lightweight shoveling position identification optimization algorithm suggested in this paper worked, we performed a number of experiments where we compared it to the YOLOv5 series algorithm and some YOLOv8 algorithms. Table 9 displays the comparison results.

During the experiment, we recorded and compared the key indicators of each algorithm in the shovel position recognition task in detail. The experimental results show that the proposed method performs well in the comprehensive comparison. The [email protected] improves by 1.1% compared to YOLOv5s, 3.7% compared to YOLOv5n, 3% compared to YOLOv5m, and 1.3% compared to YOLOv8n. However, compared with YOLOv8s, the [email protected] of this method is only 0.1% different. In addition to comparing the detection accuracy, we also consider the number of parameters, GFlops, and F1 value of the algorithm. The results show that compared with YOLOv8s, the number of parameters and GFlops of this algorithm are reduced by 6.24 M and 21.3, respectively, and the F1 value is increased by 0.6 percentage points, which means that under the same detection accuracy, our algorithm has better resource utilization efficiency. Despite slightly increasing the number of parameters compared to YOLOv5n and YOLOv8n, this algorithm successfully achieves the lightweight effect and improves the accuracy of shovel position recognition. Simultaneously, during the experiment, both the train_loss of the improved model and the validation set loss, val_loss, exhibited a downward trend throughout the iteration process, as illustrated in Figure 24. Figure shows a small gap between them. This phenomenon shows that the model did not have obvious overfitting problems during the training process.

In order to fully demonstrate the performance advantages of the improved YOLOv5s model, we use Figure 25 to completely reflect this. Figure 25a shows the [email protected] performance index of the improved model, which can more intuitively demonstrate the rationality of the improved model. The figure reveals that the improved YOLOv5s model outperforms YOLOv5s, YOLOv5n, YOLOv5m, and YOLOv8n in terms of [email protected], closely matching the performance of YOLOv8s. In addition, the P–R curve (precision–recall curve) presented in Figure 25b is a common method for evaluating the classification performance of the model. We obtain this basic curve by sorting the prediction results of the classification model under different thresholds and then calculating the relationship between the precision (P) and the recall (R). The closer the curve is to the upper right corner, the better the model performance. From Figure 25b, we can see that the improved YOLOv5s model has comparable performance to YOLOv8s. Moreover, considering that the improved model has significantly reduced the number of parameters and GFlops, it can be determined that the improvement of the model is effective.

In summary, the improved model in this paper demonstrates high performance in the shoveling position recognition task, demonstrating good detection accuracy, resource utilization efficiency, and avoiding overfitting. This model offers a viable option for unmanned loader shoveling position recognition in practical applications.

We collected the gravel pile detection dataset on the detection device test bench in Figure 26 to intuitively evaluate the effect of the improvement in this paper. We designed and built the test bench to match the ratio of 1:4.5 of a 1.2T loader. The purpose of the test bench is to replicate the perspective of an unmanned loader operation as closely as possible while also allowing for a small range of movement to capture images at various distances and angles. We selected three images from the dataset and conducted a visual comparison experiment on the detection effect of the original YOLOv5s and the improved model.

Figure 27 displays the experimental effect diagram. Figure 27 presents the detection and recognition results of both the improved model algorithm and the YOLOv5 object detection algorithm on the gravel pile dataset. The red detection boxes represent the recognition results for material pickup positions, based on the analysis in Section 3 and Section 4 of this paper, while the pink detection boxes indicate non-prioritized positions. From the figure, it is visually apparent that, compared to the original model, the improved model effectively reduces the occurrence of false detections. The comparison experiments in Section 5 also show that the improved model keeps a pretty high detection accuracy while lowering the number of parameters and GFlops that are needed for the network model. This could potentially offer a reference value solution for unmanned loaders, enabling them to achieve safe production and enhance work efficiency in material shoveling operations in the future.

6. Conclusions

Because the shape of the material pile changes during the cyclic operation of the unmanned loader, the unmanned loader cannot identify a suitable scooping position, and the current recognition network model is costly to deploy on mobile devices. Therefore, to address these issues, this paper proposes a method that combines edge curvature-based position detection features with a lightweight deep learning network model. We select gravel, a common material in loader operations, as the research object. We first analyze the positional characteristics of the loader’s scooping operation, primarily using the EDEM-Adams coupled simulation to examine the impact of the edge curvature of different piles on the scooping operation process. We conclude that the area with a convex gravel pile edge and a small degree of curvature is suitable as a scooping position, after comprehensive evaluation using radar charts and quantifying the impact. Then, based on the analysis results, we combined the proposed lightweight YOLOv5s improved model to realize the recognition and detection of the gravel material pile shoveling position. We set up a scene in a laboratory environment and used the improved model to detect the rubble pile. The results show that the model can achieve an average detection accuracy of 98% for the location of suitable material for shoveling. Compared to the original YOLOv5s model, we have reduced its parameters and GFlops by 32.5% and 55.2%, respectively. From the results, we can see that our proposed method of combining a location detection feature based on edge curvature with a lightweight model of a deep learning network is feasible.

However, this paper still has certain limitations. On the one hand, we used the EDEM-Adams coupling simulation to analyze the effect of the curvature of the material pile edge on the scooping operation. This method simplifies the intricate process of carrying out actual loader operations. In a real scooping scenario, it is difficult to accurately simulate the complex dynamic processes such as system vibration, impact, and dynamic response, which leads to differences between the simulation results and the actual dynamic process. On the other hand, although we have made lightweight improvements to the deep learning model, we have not embedded it in a mobile device. Theoretically, this has reduced the computing requirements, but in real-world mobile devices, it is unfeasible to assess the model’s optimal adaptation to hardware resources like CPU, GPU, and memory, thereby impacting the model’s overall operational efficiency and performance. In future research, we aim to address these limitations by introducing a multi-physics model for the EDEM-Adams coupled simulation problem. We will calibrate and optimize this model using actual operational data, thereby improving the simulation accuracy of complex dynamic processes in the loader operation simulation and reducing any discrepancies with the actual situation. For the problem of embedding deep learning models in mobile devices, we will conduct experiments on a variety of mobile devices, analyze the hardware characteristics, adjust the model, and continuously improve it with the help of performance monitoring to ensure that the model adapts to the hardware and improve the operating efficiency and performance of the model on mobile devices.

Author Contributions

Methodology, B.L. and S.J.; software, H.Z.; resources, B.X. and Y.X.; data curation, W.Z.; writing—original draft preparation, H.Z.; writing—review and editing, B.L. All authors have read and agreed to the published version of the manuscript.

Funding

This research is supported by the Guangxi Science and Technology Major Project, grant number AA22068064; Liuzhou Science and Technology Plan Project, grant number 2022AAB0101; Central Govemment-Guided ocal Science and Technolog Development Fund Program, grant number 2023PRJ0101.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors on request.

Acknowledgments

We appreciate the support of the Guangxi College Students Innovation and Entrepreneurship Program.

Conflicts of Interest

The authors declare no conflicts of interest.

References

Zheng, Z.; Wang, F.; Gong, G.; Yang, H.; Han, D. Intelligent technologies for construction machinery using data-driven methods. Autom. Constr. 2023, 147, 104711. [Google Scholar] [CrossRef]
You, K.; Zhou, C.; Ding, L. Deep learning technology for construction machinery and robotics. Autom. Constr. 2023, 150, 104852. [Google Scholar] [CrossRef]
Vahdatikhaki, F.; Langari, S.M.; Taher, A.; El Ammari, K.; Hammad, A. Enhancing coordination and safety of earthwork equipment operations using Multi-Agent System. Autom. Constr. 2017, 81, 267–285. [Google Scholar] [CrossRef]
Murayama, H.; Kawabe, T.; Inagawa, M.; Yoshizawa, K.; Ishibashi, M.; Takei, T.; Nagatani, K. Demonstration of Snow Removal Work by Wheel Loader in an Environment Surrounded by Obstacles. J. Robot. Mechatron. 2023, 35, 1251–1266. [Google Scholar] [CrossRef]
Dadhich, S.; Bodin, U.; Andersson, U. Key challenges in automation of earth-moving machines. Autom. Constr. 2016, 68, 212–222. [Google Scholar] [CrossRef]
Chen, G.; Wang, Y.; Li, X.; Bi, Q.; Li, X. Shovel point optimization for unmanned loader based on pile reconstruction. Comput. Civ. Infrastruct. Eng. 2024, 39, 2187–2203. [Google Scholar] [CrossRef]
Dewi, C.; Chen, R.-C.; Tai, S.-K. Evaluation of robust spatial pyramid pooling based on convolutional neural network for traffic sign recognition system. Electronics 2020, 9, 889. [Google Scholar] [CrossRef]
Duan, S.; Zhang, M.; Qiu, S.; Xiong, J.; Zhang, H.; Li, C.; Jiang, Q.; Kou, Y. Tunnel lining crack detection model based on improved YOLOv5. Tunnel. Undergr. Space Technol. 2024, 147, 105713. [Google Scholar]
Zhang, K.; Wang, C.; Yu, X.; Zheng, A.; Gao, M.; Pan, Z.; Chen, G.; Shen, Z. Research on mine vehicle tracking and detection technology based on YOLOv5. Syst. Sci. Control Eng. 2022, 10, 347–366. [Google Scholar]
Zhang, X.; Cui, B.; Wang, Z.; Zeng, W. Loader Bucket Working Angle Identification Method Based on YOLOv5s and EMA Attention Mechanism. IEEE Access 2024, 12, 105488–105496. [Google Scholar] [CrossRef]
Cao, B.-W.; Liu, X.-H.; Chen, W.; Yang, K.; Liu, D. Intelligent energy-saving operation of wheel loader based on identifiable materials. J. Mech. Sci. Technol. 2020, 34, 1081–1090. [Google Scholar] [CrossRef]
Zou, M.; Yu, J.; Lv, Y.; Lu, B.; Chi, W.; Sun, L. A novel day-to-night obstacle detection method for excavators based on image enhancement and multisensor fusion. IEEE Sens. J. 2023, 23, 10825–10835. [Google Scholar] [CrossRef]
Coetzee, C.J. Calibration of the discrete element method. Powder Technol. 2017, 310, 104–142. [Google Scholar] [CrossRef]
Yao, J.; Edson, C.P.; Yu, S.; Zhao, G.; Sun, Z.; Song, X.; Stelson, K.A. Bucket loading trajectory optimization for the automated wheel loader. ITVT 2023, 72, 6948–6958. [Google Scholar] [CrossRef]
Man, T.; Huppert, H.E.; Zhang, Z.; Galindo-Torres, S.A. Influence of cross-section shape on granular column collapses. Powder Technol. 2022, 407, 117591. [Google Scholar] [CrossRef]
Yang, G.; Xu, F. Research and analysis of Image edge detection algorithm Based on the MATLAB. Procedia Eng. 2011, 15, 1313–1318. [Google Scholar] [CrossRef]
Shaojie, W.; Liang, H.; Lee, J.; Xiangjian, B. Evaluating wheel loader operating conditions based on radar chart. Autom. Constr. 2017, 84, 42–49. [Google Scholar]
Zhao, L.; Tohti, T.; Hamdulla, A. BDC-YOLOv5: A helmet detection model employs improved YOLOv5. Signal Image Video Process. 2023, 17, 4435–4445. [Google Scholar] [CrossRef]
Chai, J.; Zeng, H.; Li, A.; Ngai, E.W. Deep learning in computer vision: A critical review of emerging techniques and application scenarios. Mach. Learn. Appl. 2021, 6, 100134. [Google Scholar] [CrossRef]
He, K.; Zhang, X.; Ren, S.; Sun, J. Spatial pyramid pooling in deep convolutional networks for visual recognition. ITPAM 2015, 37, 1904–1916. [Google Scholar] [CrossRef]
Tang, H.; Liang, S.; Yao, D.; Qiao, Y. A visual defect detection for optics lens based on the YOLOv5-C3CA-SPPF network model. Opt. Express 2023, 31, 2628–2643. [Google Scholar] [CrossRef] [PubMed]
Deng, W.; Mou, Y.; Kashiwa, T.; Escalera, S.; Nagai, K.; Nakayama, K.; Matsuo, Y.; Prendinger, H. Vision based pixel-level bridge structural damage detection using a link ASPP network. Autom. Constr. 2020, 110, 102973. [Google Scholar] [CrossRef]
Zhang, Y.; Yin, Y.; Shao, Z. An Enhanced Target Detection Algorithm for Maritime Search and Rescue Based on Aerial Images. Remote Sens. 2023, 15, 4818. [Google Scholar] [CrossRef]
Cai, Y.; Luan, T.; Gao, H.; Wang, H.; Chen, L.; Li, Y.; Sotelo, M.A.; Li, Z. YOLOv4-5D: An effective and efficient object detector for autonomous driving. IEEE Trans. Instrum. Meas. 2021, 70, 4503613. [Google Scholar] [CrossRef]
Tang, X.; Chen, Y.; Ma, Y.; Yang, W.; Zhou, H.; Huang, J. A lightweight model combining convolutional neural network and Transformer for driver distraction recognition. Eng. Appl. Artif. Intell. 2024, 132, 107910. [Google Scholar] [CrossRef]
Wang, Z.; Sun, W.; Zhu, Q.; Shi, P. Face Mask-Wearing Detection Model Based on Loss Function and Attention Mechanism. Comput. Intell. Neurosci. 2022, 2022, 2452291. [Google Scholar] [CrossRef]
Wan, F.; Sun, C.; He, H.; Lei, G.; Xu, L.; Xiao, T. YOLO-LRDD: A lightweight method for road damage detection based on improved YOLOv5s. EURASIP J. Adv. Signal Process. 2022, 2022, 98. [Google Scholar] [CrossRef]
Zhao, L.; Wang, L. A new lightweight network based on MobileNetV3. KSII Trans. Internet Inf. Syst. (TIIS) 2022, 16, 1–15. [Google Scholar]
Han, J.; Yang, Y. L-Net: Lightweight and fast object detector-based ShuffleNetV2. J. Real-Time Image Process. 2021, 18, 2527–2538. [Google Scholar] [CrossRef]
Xia, Y.; Yuan, W.; Zhang, S.; Wang, Q.; Liu, X.; Wang, H.; Wu, Y.; Yang, C.; Xu, J.; Li, L. Classification and identification of tea diseases based on improved YOLOv7 model of MobileNeXt. Sci. Rep. 2024, 14, 11799. [Google Scholar] [CrossRef]

Figure 1. Overall research methodology framework.

Figure 2. Loader execution device model, among them: 1. Bucket 2. Linkage 3. Boom 4. Rocker 5. Bucket hydraulic rod 6. Bucket hydraulic cylinder 7. Frame 8. Boom hydraulic cylinder 9. Boom hydraulic rod.

Figure 3. Loader bucket tip trajectory.

Figure 4. Crushed stone particle model.

Figure 5. Modeling convex and concave areas of shoveling gravel in a coupled EDEM and Adams simulation, among them: (a) is the simulation of shoveling the convex part of gravel; (b) simulation of shoveling concave part of gravel.

Figure 6. The lateral force of crushed stone on the bucket.

Figure 7. The gravel pile after the shovel is finished.

Figure 8. Generation of rubble piles with different drop geometries, among them: (a) cylindrical geometry; (b) rectangular geometry.

Figure 9. Resistance curve during the insertion stage.

Figure 10. Resistance curve during the lifting stage.

Figure 11. Comparison of edge curvature for simulation I. The three lines represent the different edges formed by the generation of 3375 kg of rubble. H represents the fitted edge curvature.

Figure 12. Comparison of edge curvature for simulation II. The three lines represent the different edges formed by the generation of 5063 kg of rubble. H represents the fitted edge curvature.

Figure 13. Comparison of edge curvature for simulation III. The three lines represent the different edges formed by the generation of 6750 kg of rubble. H represents the fitted edge curvature.

Figure 14. Radar chart area corresponding to different edge curvatures of 3375 kg stockpile.

Figure 15. Radar chart area corresponding to different edge curvatures of 5063 kg stockpile.

Figure 16. Radar chart area corresponding to different edge curvatures of 6750 kg stockpile.

Figure 17. Data augmentation, among them: (a) add Gaussian noise to the original image; (b) add overexposure to the original image; (c) add fog noise to the original image; (d) the original image is rotated and tilted by

\pm 5^{\circ}

to present the uneven working environment.

\pm 5^{\circ}

to present the uneven working environment.

Figure 18. Network structure diagram, where: (a) is the original YOLOv5s network structure diagram. (b) is the improved YOLOv5s network structure diagram in this paper.

Figure 19. (a) Standard convolution module; (b) ghost convolution module.

Figure 20. The bottleneck structure of ghost module, where: (a) step size is 1; (b) step size is 2.

Figure 21. RFB module.

Figure 22. Structure of CBAM.

Figure 23. C3-CBAM structure diagram, where: (a) C3-CBAM structure diagram; (b) structure diagram of CBAM bottleneck in C3.

Figure 24. Loss function curve of the improved model.

Figure 25. The performance advantage of the improved YOLOv5s model. (a) The change in [email protected] performance indicators of different model algorithms on the data set. (b) The classification performance of the improved YOLOv5s model is evaluated based on the P–R curve. In the figure, the closer the curve is to the upper right corner, the better the model performance.

Figure 26. Test bench for detection device.

Figure 27. Experimental results: (a) The image labels of this column of images used for training. (b) This column is the detection result of the original YOLOv5s model. (c) This column shows the detection results of the improved YOLOv5s model.

Table 1. Set the motion constraints of the loader actuator.

Connection Type	Kinematic Pair
Arm, frame	Revolute
Hydraulic cylinder of the arm, frame
Swing cylinder, frame
Arm, rocker arm
Rocker arm, linkage
Linkage, bucket
Frame, ground	Sliding pair
Swing cylinder, swing hydraulic rod
Arm hydraulic cylinder, arm hydraulic rod

Table 2. Loader actuator drive function settings.

Motion State	Frame Drive Function (s)	Boom Drive Function (s)	Bucket Drive Function (s)
Insertion phase	STEP (time, 0, 0.6, 2, 0.38)	None	None
Rising phase	STEP (time, 2.1, 0, 4, −0.38)	STEP (time, 2, 0, 6, 0.25)	STEP (time, 2, 0, 3.3, 0.1)
Lifting phase	STEP (time, 2.1, 0, 4, −0.38)	STEP (time, 2, 0, 6, 0.25)	STEP (time, 2, 0, 3.3, 0.1)
Resetting phase	-STEP (time, 6, 0, 7, 0.285)	None	None

Table 3. Material property parameters for crushed stone and steel.

Material	Density (kg/m³)	Shear Modulus (pa)	Poisson’s Ratio
Crushed stone	2600	5 × 10⁷	0.2
Steel	7800	8 × 10¹⁰	0.28

Table 4. Contact property parameters for crushed stone and steel.

Material	Collision Restitution Coefficient	Static Friction Coefficient	Rolling Friction Coefficient
Crushed stone	0.42	0.478	0.23
Steel	0.62	0.734	0.055

Table 5. Coupling simulation results under different stockpile edge curvatures.

Number	Total Mass	Barrel Shape	Insertion Resistance	Lifting Resistance	Mass	Curvature Value	Area
I	3375 kg	Rectangular bucket	35,946 N	93,312.3 N	473.671 kg	4.5303	0.18645
		Rectangular bucket (tilted 45°)	38,204.8 N	103,494.3 N	422 kg	5.6708	0.24211
		Circular bucket	42,761.2 N	92,422.7 N	459.159 kg	5.3136	0.20662
II	5063 kg	Rectangular bucket	35,083 N	84,912.2 N	488.375 kg	4.7858	0.19312
		Rectangular bucket (tilted 45°)	37,370 N	139,152.6 N	547.426 kg	5.3546	0.47761
		Circular bucket	58,429.1 N	128,056 N	583.812 kg	5.009	0.40117
III	6750 kg	Rectangular bucket	17,776.1 N	39,522.6 N	309.2 kg	4.1309	0.20254
		Rectangular bucket (tilted 45°)	15,006 N	145,510.3 N	303.255 kg	4.7434	1.5337
		Circular bucket	25,824.2 N	78,273.4 N	413.257 kg	4.1448	0.54658

Note: We used a loader bucket in the EDEM-Adams coupled simulation to shovel a pile of gravel materials with different edge curvature values. During the EDEM post-processing stage, we extracted the three indicators of maximum insertion resistance, maximum lifting resistance, and shoveling quality, which we then presented on the radar chart after Gaussian normalization. The area enclosed by each row of data on the radar chart was calculated, and the three rows of data with serial numbers I, II, and III were executed according to this process.

Table 6. Comparison of parameters and GFlops of five feature extraction models.

Model	Parameters	GFlops
SPP	7,225,885	16.5
SPPF	7,235,389	16.5
RFB	7,895,421	17.1
SSPCSPC	13,663,549	21.7
ASPP	15,485,725	23.1

Table 7. Comparison of experimental results for different backbone networks.

Algorithm	Parameter Quantity	GFlops	R	P	F1	[email protected]
Ghost Net	4.42 M	7	95.1%	95.6%	95.3%	97.6%
MobileNetV3	1.38 M	2.3	92.6%	91.7%	92.1%	94%
Shufflenetv2	0.84 M	1.8	88.8%	90.2%	89.5%	91.5%
MobileNeXt	4.42 M	8.4	92.7%	93.5%	93.1%	94.6%

Table 8. Ablation test results of improved YOLOv5s model.

Model	Parameter Quantity	GFlops	R	P	F1	[email protected]
YOLOv5s	7.2 M	16.5	94.9%	95.4%	95.1%	96.9%
G-v5s	4.42 M	7	95.1%	95.6%	95.3%	97.6%
GB-v5s	4.8 M	7.7	95.6%	94.9%	95.2%	97.9%
GBC-v5s	4.86 M	7.4	95.9%	95.3%	95.6%	97.8%
GBCE-v5s	4.86 M	7.4	95.3%	96.1%	95.7%	98.0%

Note: G-v5s means modifying the backbone network in YOLOv5s and introducing the Ghost Net network. GB-v5s means introducing the Ghost Net network and adding RFB in YOLOv5s. GBC-v5s means introducing the Ghost Net network, adding RFB, and modifying C3 to C3-CBAM in YOLOv5s. GBCE-v5s means introducing the Ghost Net network, adding RFB, modifying C3 to C3-CBAM, and replacing it with EIoU in YOLOv5s.

Table 9. Comparison of detection effects of different models.

Algorithm	Parameter Quantity	GFlops	R	P	F1	[email protected]
YOLOv5n	1.8 M	4.3	93.90%	93.80%	93.8%	94.3%
YOLOv5s	7.2 M	16.5	94.90%	95.40%	95.1%	96.9%
YOLOv5m	21.3 M	49.2	93.10%	94.80%	93.9%	95%
YOLOv8n	3.1 M	8.1	94.20%	94.40%	94.3%	96.7%
YOLOv8s	11.1 M	28.7	95.10%	95.10%	95.1%	98.1%
Proposed method	4.86 M	7.4	95.30%	96.10%	95.7%	98%

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

Share and Cite

MDPI and ACS Style

Zhang, H.; Jin, S.; Li, B.; Xu, B.; Xiao, Y.; Zhou, W. Research on Shoveling Position Analysis and Recognition of Unmanned Loaders for Gravel Piles. Appl. Sci. 2024, 14, 11036. https://doi.org/10.3390/app142311036

AMA Style

Zhang H, Jin S, Li B, Xu B, Xiao Y, Zhou W. Research on Shoveling Position Analysis and Recognition of Unmanned Loaders for Gravel Piles. Applied Sciences. 2024; 14(23):11036. https://doi.org/10.3390/app142311036

Chicago/Turabian Style

Zhang, Hanwen, Sun Jin, Bing Li, Bo Xu, Yuanbin Xiao, and Weixin Zhou. 2024. "Research on Shoveling Position Analysis and Recognition of Unmanned Loaders for Gravel Piles" Applied Sciences 14, no. 23: 11036. https://doi.org/10.3390/app142311036

APA Style

Zhang, H., Jin, S., Li, B., Xu, B., Xiao, Y., & Zhou, W. (2024). Research on Shoveling Position Analysis and Recognition of Unmanned Loaders for Gravel Piles. Applied Sciences, 14(23), 11036. https://doi.org/10.3390/app142311036

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Menu