CN118397483A - Fruit tree target positioning and navigation line region segmentation method based on YOLO network - Google Patents

Abstract

The invention discloses a fruit tree target positioning and navigation line area segmentation method based on a YOLO network, which belongs to the field of automatic navigation and operation of orchard machinery. The method comprises using a drone to obtain an orchard image; performing fruit tree target target detection label processing and navigation line segmentation label processing on the orchard image in sequence to obtain a drone orchard data set; according to the drone orchard data set, using a multi-task YOLO network, obtaining traversable area segmentation data and fruit tree position detection data; converting the traversable area segmentation data and the fruit tree position detection data into longitude and latitude information respectively, obtaining a navigation line and fruit tree position information for an orchard pesticide spraying robot, and completing fruit tree target positioning and navigation line area segmentation. The invention solves the problem that the current ground robot has difficulty in navigation and operation due to the complex orchard background and limited field of vision perception.

Description

Fruit tree target positioning and navigation line area segmentation method based on YOLO network

Technical Field

The invention belongs to the field of automatic navigation and operation of orchard machinery, and in particular relates to a fruit tree target positioning and navigation line area segmentation method based on a YOLO network.

Background technique

With the development of global satellite positioning systems, positioning accuracy is constantly improving, leading to the widespread integration of GNSS, GPS, and BeiDou systems in agricultural autonomous navigation operations. The navigation system is a key component of agricultural robots and directly affects the operating efficiency of agricultural robots. Agricultural machinery navigation automation is a fundamental aspect of the development of intelligent agriculture. At present, there are various automatic navigation systems for agricultural machinery, including global navigation satellite system (GNSS)/automatic identification system (INS) integrated positioning and adaptive square root cubic Kalman filter (ASRCKF). However, the use of navigation systems requires the establishment of a predefined navigation baseline, and then the driving state is adjusted in real time through the trajectory tracking algorithm to achieve precise straight line tracking operation. Obtaining the navigation baseline is a key step in achieving automatic navigation.

Machine vision is widely used in agricultural robot navigation due to its irreplaceable visual information and low hardware cost. At present, there are various visual methods for automatically extracting navigation lines, including detection, segmentation, classification, and LiDAR feature extraction. In terms of detection, methods for extracting navigation lines include the improved CS-YOLOv5 model, the YOLOv8s model improved by ASPPF, the DBSCAN integrated with YOLO-R, and the end-to-end YOLO network. In view of the problems that traditional machine vision recognition of crop rows is easily affected by light and weeds, has low recognition accuracy, and poor real-time performance. At present, segmentation algorithms are used to obtain crop rows and navigation paths. The existing technical solutions include: using a transformer-based semantic segmentation model; using a new autonomous navigation stack with a three-dimensional camera and an inertial measurement unit (IMU), which integrates ResNet-50 (IMU) and an attention mechanism module; using a vision-based fusion vegetation index and segmentation method; using an improved multi-scale efficient residual decomposition convolutional neural network (MS-ERFNet) model; using a semi-supervised learning model and an improved UNet network.

Since 3D point clouds have the advantages of high multi-dimensional information and low photosensitivity, a large number of studies have been carried out on the use of LiDAR to obtain navigation routes. Existing technical solutions have proposed: a line segment-based method for extracting low-curvature effective road free space (RFS); an FGSeg algorithm to effectively distinguish between horizontal and slope terrains, which is compatible with a variety of LiDAR sensors used in agricultural field environments; a LiDAR histogram method for comprehensive detection of traversable road areas, obstacles and water hazards; and the extraction of field straight roads between farm fields based on agricultural vehicle-mounted LiDAR.

Fruit tree recognition and fruit tree target positioning are key technologies for spraying robots, and are increasingly attracting the attention of researchers and developers. Existing technical solutions include: a straight and curved seedling row recognition method based on sub-region growth and outlier removal; a hybrid autonomous robotic weeding system based on a fully integrated three-axis platform and a visual system mounted on a mobile probe vehicle, and a pre-trained deep neural network is used to obtain spatial, color and depth information for classification of soil, main crops and objects to be removed; a new plant detection model is constructed using a CBAM module, a BiFPN structure and a bilinear interpolation algorithm, which can effectively learn depth information and distinguish plants in cotton seedlings under various complex growth states; a new method for motion blurred image restoration based on a wide receptive field attention network (WRA-Net), and on this basis, how to improve the segmentation accuracy of crops and weeds in motion blurred images is studied. Although significant achievements have been made in the field of automatic navigation and target detection, there are still several outstanding issues that need to be addressed, especially in orchard planting. These issues include: when autonomous navigation relies only on a single feature, such as traversable areas such as tree paths, its robustness and reliability will be greatly reduced. Due to the complex background of large-scale orchards and alternating growth patterns of fruit trees, the real-time perception capability of ground vehicles is limited.

Summary of the invention

In view of the above-mentioned deficiencies in the prior art, the present invention provides a fruit tree target positioning and navigation line area segmentation method based on the YOLO network, which solves the navigation and operation difficulties of current ground robots caused by complex fruit orchard backgrounds and limited field of view perception.

In order to achieve the above-mentioned invention object, the technical solution adopted by the present invention is: a fruit tree target positioning and navigation line area segmentation method based on YOLO network, comprising the following steps:

S1. Use drones to obtain orchard images;

S2, perform fruit tree target detection label processing and navigation line segmentation label processing on the orchard image in sequence to obtain the drone orchard dataset;

S3. Based on the drone orchard dataset, the multi-task YOLO network is used to obtain the traversable area segmentation data and fruit tree location detection data;

S4. Convert the traversable area segmentation data and the fruit tree position detection data into longitude and latitude information respectively, obtain the navigation line and fruit tree position information for the orchard pesticide spraying robot, and complete the fruit tree target positioning and navigation line area segmentation.

Furthermore, the step S2 is specifically as follows:

S201, screening the orchard image to obtain screened effective orchard image data;

S202: Based on the existing orchard image data, the fruit tree target anchor frame is annotated, and the fruit tree plants and field lanes are segmented and labeled to obtain the drone orchard dataset.

Furthermore, the multi-task YOLO network includes a backbone network backbone, a neck connected to the backbone network backbone, and a prediction head head connected to the neck neck.

Furthermore, the backbone network backbone includes a Focus structure, a first Conv2d convolution, a first C2f gradient shunt module, a second Conv2d convolution, a second C2f gradient shunt module, a third Conv2d convolution, a third C2f gradient shunt module, a fourth Conv2d convolution and an SPP structure connected in sequence; the second C2f gradient shunt module, the third C2f gradient shunt module and the SPP structure are all connected to the neck.

Furthermore, the neck includes a fourth C2f gradient shunt module connected to the SPP structure, a fifth Conv2d convolution connected to the fourth C2f gradient shunt module, a first Upsample upsampling connected to the fifth Conv2d convolution, a first Concat splicing connected to the first Upsample upsampling and the third C2f gradient shunt module respectively, a fifth C2f gradient shunt module connected to the first Concat splicing, a sixth Conv2d convolution connected to the fifth C2f gradient shunt module and the prediction head respectively, a second Upsample upsampling connected to the sixth Conv2d convolution, and a second Concat splicing connected to the second Upsample upsampling, the second C2f gradient shunt module and the prediction head respectively.

Furthermore, the prediction head includes a navigation line region segmentation network for segmenting a feasible area of a waterway line and a fruit tree target positioning network for target detection of fruit tree targets.

Furthermore, the navigation line area segmentation network includes a seventh Conv2d convolution connected to the second Concat splicing, a sixth C2f gradient shunt module connected to the seventh Conv2d convolution, a third Upsample upsampling connected to the sixth C2f gradient shunt module, an eighth Conv2d convolution connected to the third Upsample upsampling, a fourth Upsample upsampling connected to the eighth Conv2d convolution, a seventh C2f gradient shunt module connected to the fourth Upsample upsampling, a ninth Conv2d convolution connected to the seventh C2f gradient shunt module, a fifth Upsample upsampling connected to the ninth Conv2d convolution, a tenth Conv2d convolution connected to the fifth Upsample upsampling, and a Seg_Loss loss function layer connected to the tenth Conv2d convolution.

Furthermore, the fruit tree target positioning network includes an eighth C2f gradient shunt module connected to the second Concat splicing, an eleventh Conv2d convolution connected to the eighth C2f gradient shunt module, a third Concat splicing connected to the eleventh Conv2d convolution and the sixth Conv2d convolution respectively, a twelfth Conv2d convolution connected to the third Concat splicing, a fourth Concat splicing connected to the twelfth Conv2d convolution, a ninth C2f gradient shunt module connected to the fourth Concat splicing, and three detection heads connected to the ninth C2f gradient shunt module respectively; each detection head includes a first branch connected by a Conv convolution, a Conv2d convolution and a Bbox_Loss loss layer, and a second branch connected by a Conv convolution, a Conv2d convolution and a Class_Loss loss layer.

Furthermore, the loss function of the multi-task YOLO network in step S3 is:

L _all ＝γ ₁ L _det +γ ₂ L _da-seg

L _det =α ₁ L _class +α ₂ L _obj +α ₃ L _box

L _da-seg ＝L _ce

Among them, L _all is the loss function of the multi-task YOLO network; γ ₁ is the detection loss weight; L _det is the detection loss; γ ₂ is the segmentation loss weight; L _da-seg is the segmentation loss; α ₁ , α ₂ and α ₃ are all hyperparameters; L _class is the classification loss, specifically the focal loss function; L _obj is the target loss, specifically the focal loss function; L _box is the bounding box loss, specifically the CloU loss function; L _ce is the cross entropy loss with Logits.

Furthermore, the step S4 is specifically as follows:

S401, converting the fruit tree location detection data into longitude and latitude information to obtain the fruit tree location information;

S402, extracting edge line data corresponding to color blocks in the passable area according to the passable area segmentation data;

S403, filtering the noise points of the edge line data corresponding to the color block in the passable area to obtain the navigation line pixel data;

S404, converting the navigation line pixel data into longitude and latitude information, obtaining the navigation line for the orchard pesticide spraying robot, and completing the fruit tree target positioning and navigation line area segmentation.

The beneficial effects of the present invention are as follows: the YOLO algorithm requires a high-quality data set, and the present invention can screen out a high-quality image data set, and at the same time, the algorithm can learn the orchard target and orchard navigation line data information required for orchard operations in a supervised learning manner; the present invention can achieve better data learning through three structures, and the backbone is the main component of the model, which is usually a convolutional neural network (CNN) or a residual neural network (ResNet). The backbone is responsible for extracting the features of the input image for subsequent processing and analysis. The backbone usually has many layers and many parameters, and can extract high-level feature representations of the image. The neck is an intermediate layer connecting the backbone and the head. The main function of the neck is to reduce the dimension or adjust the features from the backbone to better meet the task requirements. The neck can use a convolutional layer, a pooling layer or a fully connected layer. The head is the last layer of the model, which is usually a classifier or a regressor. The head generates the final output result by inputting the features processed by the neck; the image information predicted by the algorithm, combined with the longitude and latitude information in the image, can be directly translated into the actual longitude and latitude information that can be directly executed by the orchard pesticide spraying robot. It can greatly reduce the manual workload and improve the work efficiency and convenience. There is no need for on-site survey of artificial orchards. It only requires drone flight planning, algorithm recognition, and information conversion to obtain the final executable task information. The present invention integrates the C2f gradient shunting module and the anchor-free module, which can simultaneously perform tree-area segmentation and fruit tree detection tasks, enhance the algorithm's fruit tree target detection and tree-road recognition capabilities, optimize resource utilization, and improve task execution efficiency; the limited perception capabilities of ground robots can be solved by utilizing the perspective of aerial drones; the multi-task YOLO network proposed in the present invention has good performance and is suitable for automatic navigation and operation of orchard machinery.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a flow chart of the method of the present invention.

FIG2 is a diagram of the multi-task YOLO network structure of the present invention.

Detailed ways

The specific implementation modes of the present invention are described below to facilitate the understanding of the present invention by those skilled in the art. However, it should be clear that the present invention is not limited to the scope of the specific implementation modes. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations utilizing the concept of the present invention are protected.

As shown in FIG1 , in one embodiment of the present invention, a method for fruit tree target positioning and navigation line area segmentation based on a YOLO network includes the following steps:

S1. Use drones to obtain orchard images;

S2, perform fruit tree target detection label processing and navigation line segmentation label processing on the orchard image in sequence to obtain the drone orchard dataset;

S3. Based on the drone orchard dataset, the multi-task YOLO network is used to obtain the traversable area segmentation data and fruit tree location detection data;

S4. Convert the traversable area segmentation data and the fruit tree position detection data into longitude and latitude information respectively, obtain the navigation line and fruit tree position information for the orchard pesticide spraying robot, and complete the fruit tree target positioning and navigation line area segmentation.

In this embodiment, the purpose of the present invention is to solve the problem that the current ground robot has limited field of vision perception due to the complex background of the orchard. Based on the extended data captured by the drone perspective, a new multi-task YOLO network is proposed for fruit tree target positioning and navigation line area segmentation, and the latitude and longitude information of the actual target and navigation line coordinates are obtained. First, the improved multi-task YOLO algorithm segments the orchard's spraying equipment traversable area in the drone image and detects the position of the fruit tree target. Subsequently, the segmented traversable area is subjected to navigation line extraction, including a series of operations such as regional edge extraction, contour regionalization, filtering operation, and navigation line extraction to obtain the current route. In order to generate executable information, software is used to convert the identified tree-to-tree route pixel points and fruit tree target positioning pixels into actual latitude and longitude information for the spraying vehicle to execute. The improved multi-task YOLO network uses C2f and anchor-free modules to enhance the multi-task YOLO algorithm. The multi-task YOLO algorithm is enhanced by using C2f and anchor-free modules to improve the detection performance of fruit tree targets and tree-to-tree route extraction in orchards with complex backgrounds.

The step S2 is specifically as follows:

S201, screening the orchard image to obtain screened effective orchard image data;

S202: Based on the existing orchard image data, the fruit tree target anchor frame is annotated, and the fruit tree plants and field lanes are segmented and labeled to obtain the drone orchard dataset.

In this embodiment, the algorithm data set of the present invention is collected by a DJI drone, and the data collected by the drone is labeled, one is the fruit tree target detection label processing, and the other is the navigation line segmentation label processing. LabelMe is used to annotate the fruit tree targets and field channels. All relevant information, including target object categories, target frame coordinates, fruit tree segmentation and field channel segmentation information, are stored in *.JSON format. The labeled data set is divided into a training set, a validation set, and a test set in a ratio of 8:1:1, wherein the validation set and the test set are mainly used to evaluate the performance of the network in the present invention.

As shown in FIG2 , the multi-task YOLO network includes a backbone network backbone, a neck connected to the backbone network backbone, and a prediction head head connected to the neck neck.

The backbone network backbone includes a Focus structure, a first Conv2d convolution, a first C2f gradient shunt module, a second Conv2d convolution, a second C2f gradient shunt module, a third Conv2d convolution, a third C2f gradient shunt module, a fourth Conv2d convolution and an SPP structure connected in sequence; the second C2f gradient shunt module, the third C2f gradient shunt module and the SPP structure are all connected to the neck.

The neck includes a fourth C2f gradient shunt module connected to the SPP structure, a fifth Conv2d convolution connected to the fourth C2f gradient shunt module, a first Upsample upsampling connected to the fifth Conv2d convolution, a first Concat splicing connected to the first Upsample upsampling and the third C2f gradient shunt module respectively, a fifth C2f gradient shunt module connected to the first Concat splicing, a sixth Conv2d convolution connected to the fifth C2f gradient shunt module and a prediction head respectively, a second Upsample upsampling connected to the sixth Conv2d convolution, and a second Concat splicing connected to the second Upsample upsampling, the second C2f gradient shunt module and the prediction head respectively.

The prediction head includes a navigation line region segmentation network for segmenting the feasible region of the waterway line and a fruit tree target positioning network for object detection of the fruit tree target.

The navigation line area segmentation network includes a seventh Conv2d convolution connected to the second Concat splicing, a sixth C2f gradient shunt module connected to the seventh Conv2d convolution, a third Upsample upsampling connected to the sixth C2f gradient shunt module, an eighth Conv2d convolution connected to the third Upsample upsampling, a fourth Upsample upsampling connected to the eighth Conv2d convolution, a seventh C2f gradient shunt module connected to the fourth Upsample upsampling, a ninth Conv2d convolution connected to the seventh C2f gradient shunt module, a fifth Upsample upsampling connected to the ninth Conv2d convolution, a tenth Conv2d convolution connected to the fifth Upsample upsampling, and a Seg_Loss loss function layer connected to the tenth Conv2d convolution.

The fruit tree target positioning network includes an eighth C2f gradient shunt module connected to the second Concat splicing, an eleventh Conv2d convolution connected to the eighth C2f gradient shunt module, a third Concat splicing connected to the eleventh Conv2d convolution and the sixth Conv2d convolution respectively, a twelfth Conv2d convolution connected to the third Concat splicing, a fourth Concat splicing connected to the twelfth Conv2d convolution, a ninth C2f gradient shunt module connected to the fourth Concat splicing, and three detection heads connected to the ninth C2f gradient shunt module respectively; each detection head includes a first branch connected by a Conv convolution, a Conv2d convolution and a Bbox_Loss loss layer, and a second branch connected by a Conv convolution, a Conv2d convolution and a Class_Loss loss layer.

In this embodiment, the multi-task network enables the fruit tree target robot to achieve unmanned navigation and accurate target identification. The robot uses the drone with deep visual perception to understand the scene, provide navigation data for the motion planning module, and provide information on the location of the fruit trees and the drivable area. The multi-task YOLO network is selected as the basic framework and enhanced to simultaneously perform fruit tree target detection and field lane line area segmentation.

In this embodiment, the improved multi-task YOLO network consists of a backbone, a neck and a head, wherein the backbone consists of a Focus structure, four Conv2d, three C2f and a SPP structure. The neck adopts a PAN/FAN structure, and performs up and down sampling through multiple Concat and Umsample, in which C2f and Conv2d are connected. The head part consists of two parts, one for the segmentation of the feasible area of the waterway line, and the other for the target detection of the fruit tree target. Among them, the head of the segmentation part is composed of 4 Conv2d, 3 Cpsample, 2 C2f and a Seg Loss function. The head of the target detection part is composed of 2 common C2f, 2 Conv2d and 2 Concat, extending 3 detection heads of different scales, that is, an anchor-free structure. Finally, the target detection results and passable areas of the target tree are predicted and output for subsequent waterway line extraction. The multi-task YOLO network shares an encoder consisting of a backbone network and a neck, and combines two decoders to solve the problems of fruit tree target detection and fruit tree target robot navigation line extraction respectively.

The loss function of the multi-task YOLO network in step S3 is:

L _all ＝γ ₁ L _det +γ ₂ L _da-seg

L _det =α ₁ L _class +α ₂ L _obj +α ₃ L _box

L _da-seg ＝L _ce

Among them, L _all is the loss function of the multi-task YOLO network; γ ₁ is the detection loss weight; L _det is the detection loss; γ ₂ is the segmentation loss weight; L _da-seg is the segmentation loss; α ₁ , α ₂ and α ₃ are all hyperparameters; L _class is the classification loss, specifically the focal loss function; L _obj is the target loss, specifically the focal loss function; L _box is the bounding box loss, specifically the CloU loss function; L _ce is the cross entropy loss with Logits.

In this embodiment, a suitable hardware configuration and software environment for training the detection network is selected. The platform is a desktop computer running Ubuntu 22.04, equipped with a high-performance Intel I9-13900K CPU, 64GB memory, and NVIDIA 4090 24GB GPU. The experimental code uses CUDA 11.1, cuDNN 8.6.0, Python 3.8, PyTorch1.8.0, and VSCode. The network input size is set to 640×640 and the batch size is 16. During the training process, the Adam stochastic gradient descent algorithm was used, the initial learning rate was set to 0.01, the weight decay coefficient was 0.0005, and the momentum was 0.937. In addition, the research model is loaded with official pre-trained weights to accelerate the training process and improve model performance.

The multi-task YOLO network contains two output heads, one for detection and one for segmentation, so its loss function contains two parts: detection loss and segmentation loss. The detection loss consists of classification loss, target loss and bounding box loss. The classification loss and target loss use focal loss functions to reduce the loss of well-classified samples and make the network more focused on samples that are difficult to classify. The bounding box loss uses LCIoU, which comprehensively considers the distance, overlap, similarity and aspect ratio between the predicted box and the true box. The loss of drivable area segmentation uses cross entropy loss with Logits, which aims to minimize the classification error between the network output pixels and the target pixels. The final total loss is the weighted sum of these three losses, and hyperparameters need to be adjusted to balance the different subtasks of multi-task perception.

In this embodiment, in order to improve the performance of the model, a variety of data enhancement techniques and methods are used. The k-means clustering algorithm is used to extract prior anchor points from the data set to enhance the detector's prior knowledge of objects in the fruit tree target scene. In terms of model optimization, Adam is selected as the optimizer, and the initial learning rate, γ ₁ and γ ₂ are set to 0.001, 0.937 and 0.999 respectively. The preheating and cosine annealing strategies are used to adjust the learning rate to speed up the convergence speed and improve the model performance. Data enhancement is used to increase image diversity and ensure that the model is robust in different environments.

The step S4 is specifically as follows:

S401, converting the fruit tree location detection data into longitude and latitude information to obtain the fruit tree location information;

S402, extracting edge line data corresponding to color blocks in the passable area according to the passable area segmentation data;

S403, filtering the noise points of the edge line data corresponding to the color block in the passable area to obtain the navigation line pixel data;

S404, converting the navigation line pixel data into longitude and latitude information to obtain the navigation line for the orchard pesticide spraying robot, thereby completing the fruit tree target positioning and the navigation line area segmentation.

In this embodiment, for navigation line extraction, it is necessary to extract navigation lines based on the unmanned vehicle passable area predicted by the multi-task YOLO network. The main steps are: image input, multi-task YOLO algorithm predicts the passable area, segmented area color block extraction and edge extraction, denoising and navigation line extraction.

Since the multi-task YOLO network predicts the color blocks in the passable area, it is necessary to extract the edge area corresponding to the color block for extracting the navigation line. The area extraction algorithm used in this embodiment involves the following steps:

Step 1: Set the HSV values of blue pixels to the range threshold (110, 50, 50) to (130, 255, 255). Set the HSV values of red pixels to the range threshold (0, 100, 100) to (10, 255, 255).

Step 2: Image processing: Binarize the image to adopt a pixel value format represented by only 0 and 1. Pixel values within the threshold range become 0, while pixel values outside the threshold range become 1.

Step 3: Image traversal: Use a kernel with zeros on the left and ones on the right. Traverse the image starting from the upper left corner, from left to right and from top to bottom. The first point encountered during the traversal that is aligned with the specified kernel is determined as the initial point of the outer boundary of the geometric entity.

Step 4: Boundary Tracking A specific direction for boundary tracking is established. A loop is then started to search for points to determine the boundaries of the geometry.

Step 5: Calculate the corresponding navigation line in the middle of the color block based on the geometric boundary.

The multi-task YOLO network can generate a lot of noise and prediction errors when processing drone images, especially when predicting color blocks. To improve the accuracy of navigation line extraction, a filtering method based on the number of pixels is used to eliminate algorithmic errors, thereby retaining color blocks with a large number of pixels such as fruit trees and field lanes. After processing the color blocks, denoising is performed and the color blocks are fitted using a rectangular matrix, and finally the navigation lines are extracted. This method improves the robustness and efficiency of navigation line extraction and provides a novel solution compared to traditional single-task methods based on crop rows or field lanes.

Verification of algorithm prediction effect:

In order to evaluate the performance of the multi-task YOLO network in fruit tree target detection and navigation line segmentation, images taken by drones were used as test sets, and the model was trained for 250 cycles. During the training process, it was observed that the loss of the model gradually decreased, while mIoU and Acc increased significantly after the 50th cycle. At the 250th cycle, the model converged, and the accuracy of fruit tree target detection and navigation line segmentation reached 77.80% and 84.37%, respectively. These results show that the model has high accuracy in fruit tree target detection and navigation line segmentation, which helps to reduce agricultural economic losses and promote the application of new methods in this field.

The multi-task YOLO network trained according to the alternating update strategy performed well in the tasks of fruit tree target detection and navigation line area segmentation, with accuracy, recall, mAP50, mIoU and accuracy reaching 84.37%, 75.30%, 81.01%, 77.80% and 86.35% respectively. In this embodiment, the baseline algorithm YOLOP is optimized by introducing the C2f module and the anchor-free module to improve the performance of the model. The C2f module combines low-level and high-level feature maps to enhance the model's ability to capture subtle gradient flow information. The anchor-free module reduces the reliance on the anchor framework and further optimizes the model. The results on the test set show that these improvements effectively improve the accuracy and mIoU of the multi-task YOLO.

Multi-task YOLO performs well in the navigation line area segmentation task. The comparative experimental results show that although it may not be as good as PP-liteseg, it has advantages as a multi-task integrated algorithm. In the complex background of drone orchard images, multi-task YOLO outperforms Deeplabv3 and SegNet models through the C2f module and a powerful feature extraction backbone network, effectively alleviating the difficulty of navigation line extraction. Compared with other algorithms in the YOLO series, the improved multi-task YOLO has significantly improved in accuracy and mAP50, proving its importance as an optimization solution for automatic pesticide spraying robots in orchards.

The present invention uses orchard images and fruit tree target datasets collected by drones to propose an enhanced multi-task YOLO network for navigation line area extraction and fruit tree target detection, providing navigation for unmanned weeders. The model structure is improved through the C2f module, anchor dependency is reduced, and detection and feature extraction capabilities are improved. Experimental results show that the improved model improves training accuracy and mIoU by 4.27% and 2.50%, respectively. In addition, the present invention also extracts navigation lines based on crop rows and field roads through a series of image processing operations and converts them into actual longitude and latitude information. Compared with the positioning information of professional equipment, the error is only 5.472cm.

The advantage of adopting the above structure is that the previous research on navigation and target detection mainly focused on real-time visual navigation of vehicles or inter-row extraction based on target detection. However, due to the low viewpoint, ground robots have limitations in perception. The existing target object detection and path extraction methods are only effective for sparse orchards. For orchards with complex backgrounds and dense and alternating fruit trees, effective extraction is not possible. Therefore, the present invention combines the perspective of unmanned aerial vehicles (UAVs) with the new multi-task YOLO algorithm, and proposes a navigation method for managing actual longitude and latitude coordinate information based on orchard navigation area extraction and fruit tree target acquisition that can be used for orchard spraying vehicles. First, the improved multi-task YOLO network segments the orchard spraying equipment traversable area in the drone image and detects the position of the fruit tree target. Subsequently, the segmented traversable area is subjected to navigation line extraction, including a series of operations such as regional edge extraction, contour regionalization, filtering operation, and navigation line extraction to obtain the current route. In order to generate executable information, software is used to convert the identified inter-tree route pixels and fruit tree target positioning pixels into actual latitude and longitude information for the spraying vehicle to execute. The improved multi-task YOLO network uses C2f and anchor-free modules to enhance the multi-task YOLO. The research results of the present invention show that compared with the original model, the training accuracy of the improved multi-task YOLO network is improved by 4.27% and the mIoU is improved by 2.50%. These research results show that the navigation method and target positioning method proposed in the present invention can effectively solve the limited perception ability and positioning problems of ground robots.

Claims (10)

1. A fruit tree target positioning and navigation line area segmentation method based on YOLO network, characterized in that it includes the following steps: S1. Use drones to obtain orchard images; S2, perform fruit tree target detection label processing and navigation line segmentation label processing on the orchard image in sequence to obtain the drone orchard dataset; S3. Based on the drone orchard dataset, the multi-task YOLO network is used to obtain the traversable area segmentation data and fruit tree location detection data; S4. Convert the traversable area segmentation data and the fruit tree position detection data into longitude and latitude information respectively, obtain the navigation line and fruit tree position information for the orchard pesticide spraying robot, and complete the fruit tree target positioning and navigation line area segmentation. 2. According to the fruit tree target positioning and navigation line area segmentation method based on the YOLO network of claim 1, it is characterized in that the step S2 is specifically: S201, screening the orchard image to obtain screened effective orchard image data; S202: Based on the existing orchard image data, the fruit tree target anchor frame is annotated, and the fruit tree plants and field lanes are segmented and labeled to obtain the drone orchard dataset. 3. according to the fruit tree target positioning and navigation line region segmentation method based on the YOLO network of claim 1, it is characterized in that the multi-task YOLO network comprises a backbone network backbone, a neck connected to the backbone network backbone and a prediction head head connected to the neck neck. 4. According to the fruit tree target positioning and navigation line area segmentation method based on the YOLO network in claim 3, it is characterized in that the backbone network backbone includes a Focus structure, a first Conv2d convolution, a first C2f gradient shunt module, a second Conv2d convolution, a second C2f gradient shunt module, a third Conv2d convolution, a third C2f gradient shunt module, a fourth Conv2d convolution and an SPP structure connected in sequence; the second C2f gradient shunt module, the third C2f gradient shunt module and the SPP structure are all connected to the neck. 5. According to the fruit tree target positioning and navigation line area segmentation method based on the YOLO network in claim 4, it is characterized in that the neck includes a fourth C2f gradient shunt module connected to the SPP structure, a fifth Conv2d convolution connected to the fourth C2f gradient shunt module, a first Upsample upsampling connected to the fifth Conv2d convolution, a first Concat splicing connected to the first Upsample upsampling and the third C2f gradient shunt module respectively, a fifth C2f gradient shunt module connected to the first Concat splicing, a sixth Conv2d convolution connected to the fifth C2f gradient shunt module and the prediction head respectively, a second Upsample upsampling connected to the sixth Conv2d convolution, and a second Concat splicing connected to the second Upsample upsampling, the second C2f gradient shunt module and the prediction head respectively. 6. According to the fruit tree target positioning and navigation line area segmentation method based on the YOLO network in claim 5, it is characterized in that the prediction head includes a navigation line area segmentation network for segmenting the feasible area of the waterway line and a fruit tree target positioning network for target detection of the fruit tree target. 7. According to the fruit tree target positioning and navigation line area segmentation method based on the YOLO network in claim 6, it is characterized in that the navigation line area segmentation network includes a seventh Conv2d convolution connected to the second Concat splicing, a sixth C2f gradient shunt module connected to the seventh Conv2d convolution, a third Upsample upsampling connected to the sixth C2f gradient shunt module, an eighth Conv2d convolution connected to the third Upsample upsampling, a fourth Upsample upsampling connected to the eighth Conv2d convolution, a seventh C2f gradient shunt module connected to the fourth Upsample upsampling, a ninth Conv2d convolution connected to the seventh C2f gradient shunt module, a fifth Upsample upsampling connected to the ninth Conv2d convolution, a tenth Conv2d convolution connected to the fifth Upsample upsampling, and a Seg_Loss loss function layer connected to the tenth Conv2d convolution. 8. According to the fruit tree target positioning and navigation line area segmentation method based on the YOLO network in claim 6, it is characterized in that the fruit tree target positioning network includes an eighth C2f gradient shunt module connected to the second Concat splicing, an eleventh Conv2d convolution connected to the eighth C2f gradient shunt module, a third Concat splicing connected to the eleventh Conv2d convolution and the sixth Conv2d convolution respectively, a twelfth Conv2d convolution connected to the third Concat splicing, a fourth Concat splicing connected to the twelfth Conv2d convolution, a ninth C2f gradient shunt module connected to the fourth Concat splicing, and three detection heads connected to the ninth C2f gradient shunt module respectively; each detection head includes a first branch connected by a Conv convolution, a Conv2d convolution and a Bbox_Loss loss layer and a second branch connected by a Conv convolution, a Conv2d convolution and a Class_Loss loss layer. 9. According to the fruit tree target positioning and navigation line area segmentation method based on the YOLO network of claim 1, it is characterized in that the loss function of the multi-task YOLO network in the step S3 is: L _all ＝γ ₁ L _det +γ ₂ L _da-seg L _det =α ₁ L _class +α ₂ L _obj +α ₃ L _box L _da-seg ＝L _ce Among them, L _all is the loss function of the multi-task YOLO network; γ ₁ is the detection loss weight; L _det is the detection loss; γ ₂ is the segmentation loss weight; L _da-seg is the segmentation loss; α ₁ , α ₂ and α ₃ are all hyperparameters; L _class is the classification loss, specifically the focal loss function; L _obj is the target loss, specifically the focal loss function; L _box is the bounding box loss, specifically the CloU loss function; L _ce is the cross entropy loss with Logits. 10. According to the fruit tree target positioning and navigation line area segmentation method based on the YOLO network of claim 1, it is characterized in that the step S4 is specifically: S401, converting the fruit tree location detection data into longitude and latitude information to obtain the fruit tree location information; S402, extracting edge line data corresponding to color blocks in the passable area according to the passable area segmentation data; S403, filtering the noise points of the edge line data corresponding to the color block in the passable area to obtain the navigation line pixel data; S404, converting the navigation line pixel data into longitude and latitude information, obtaining the navigation line for the orchard pesticide spraying robot, and completing the fruit tree target positioning and navigation line area segmentation.