Agriculture 2024, 14(12), 2268; https://doi.org/10.3390/agriculture14122268 (registering DOI) - 11 Dec 2024
Abstract
Accurate and efficient tomato detection is one of the key techniques for intelligent automatic picking in the area of precision agriculture. However, under the facility scenario, existing detection algorithms still have challenging problems such as weak feature extraction ability for occlusion conditions and
[...] Read more.
Accurate and efficient tomato detection is one of the key techniques for intelligent automatic picking in the area of precision agriculture. However, under the facility scenario, existing detection algorithms still have challenging problems such as weak feature extraction ability for occlusion conditions and different fruit sizes, low accuracy on edge location, and heavy model parameters. To address these problems, this paper proposed D3-YOLOv10, a lightweight YOLOv10-based detection framework. Initially, a compact dynamic faster network (DyFasterNet) was developed, where multiple adaptive convolution kernels are aggregated to extract local effective features for fruit size adaption. Additionally, the deformable large kernel attention mechanism (D-LKA) was designed for the terminal phase of the neck network by adaptively adjusting the receptive field to focus on irregular tomato deformations and occlusions. Then, to further improve detection boundary accuracy and convergence, a dynamic FM-WIoU regression loss with a scaling factor was proposed. Finally, a knowledge distillation scheme using semantic frequency prompts was developed to optimize the model for lightweight deployment in practical applications. We evaluated the proposed framework using a self-made tomato dataset and designed a two-stage category balancing method based on diffusion models to address the sample class-imbalanced issue. The experimental results demonstrated that the D3-YOLOv10 model achieved an
(This article belongs to the Special Issue Multi- and Hyper-Spectral Imaging Technologies for Crop Monitoring—2nd Edition)
►
Show Figures
Figure 1
Figure 1
<p>The workflow diagram for this paper.</p> Full article ">Figure 2
<p>Tomato occlusion in the facility scenario, including the unoccluded, mutual occlusion, leaf occlusion, and facility occlusion.</p> Full article ">Figure 3
<p>Structure diagram of the two-stage class balancing method based on the diffusion model.</p> Full article ">Figure 4
<p><b>Overview of the experimental framework. Stage 1</b>: With the teacher model’s guidance, the learnable frequency prompts interact with the frequency bands. <b>Stage 2</b>: The feature maps distilled from both the student and teacher are initially transformed into the frequency domain. The frequency prompts from Stage 1 are then applied, with the frozen prompts multiplied by the teacher’s frequency bands to generate points of interest (PoIs). Finally, the spatial weights for each channel are determined by the teacher and student spatial gates. Process (1) in the figure identifies the distillation locations, while Process (2) measures the distillation extent.</p> Full article ">Figure 5
<p>Architecture of the DyFasterNet module: (<b>a</b>) DyFasterNet; (<b>b</b>) Dynamic convolution.</p> Full article ">Figure 6
<p>Deformable large kernel attention mechanism structure.</p> Full article ">Figure 7
<p>The proposed Inner-FM-WIoU.</p> Full article ">Figure 8
<p>YOLOv10s model performance for class-balanced datasets vs. class-imbalanced datasets.</p> Full article ">Figure 9
<p>The loss curve and <math display="inline"><semantics> <mrow> <mi>m</mi> <mi>A</mi> <msub> <mi>P</mi> <mrow> <mn>0.5</mn> </mrow> </msub> </mrow> </semantics></math> under different scaling ratios.</p> Full article ">Figure 10
<p>Examples of detection capabilities of comparative models in facility environment.</p> Full article ">Figure 11
<p>Robustness experiment in various scenarios: Area (<b>a</b>) demonstrates the performance of the model in a facility agriculture environment with occlusion. Areas (<b>b</b>,<b>c</b>,<b>e</b>) show tomatoes partially occluded by branches or leaves. While area (<b>d</b>) highlights small target recognition. Finally, area (<b>f</b>) presents cases of mutual occlusion between fruits.</p> Full article ">
<p>The workflow diagram for this paper.</p> Full article ">Figure 2
<p>Tomato occlusion in the facility scenario, including the unoccluded, mutual occlusion, leaf occlusion, and facility occlusion.</p> Full article ">Figure 3
<p>Structure diagram of the two-stage class balancing method based on the diffusion model.</p> Full article ">Figure 4
<p><b>Overview of the experimental framework. Stage 1</b>: With the teacher model’s guidance, the learnable frequency prompts interact with the frequency bands. <b>Stage 2</b>: The feature maps distilled from both the student and teacher are initially transformed into the frequency domain. The frequency prompts from Stage 1 are then applied, with the frozen prompts multiplied by the teacher’s frequency bands to generate points of interest (PoIs). Finally, the spatial weights for each channel are determined by the teacher and student spatial gates. Process (1) in the figure identifies the distillation locations, while Process (2) measures the distillation extent.</p> Full article ">Figure 5
<p>Architecture of the DyFasterNet module: (<b>a</b>) DyFasterNet; (<b>b</b>) Dynamic convolution.</p> Full article ">Figure 6
<p>Deformable large kernel attention mechanism structure.</p> Full article ">Figure 7
<p>The proposed Inner-FM-WIoU.</p> Full article ">Figure 8
<p>YOLOv10s model performance for class-balanced datasets vs. class-imbalanced datasets.</p> Full article ">Figure 9
<p>The loss curve and <math display="inline"><semantics> <mrow> <mi>m</mi> <mi>A</mi> <msub> <mi>P</mi> <mrow> <mn>0.5</mn> </mrow> </msub> </mrow> </semantics></math> under different scaling ratios.</p> Full article ">Figure 10
<p>Examples of detection capabilities of comparative models in facility environment.</p> Full article ">Figure 11
<p>Robustness experiment in various scenarios: Area (<b>a</b>) demonstrates the performance of the model in a facility agriculture environment with occlusion. Areas (<b>b</b>,<b>c</b>,<b>e</b>) show tomatoes partially occluded by branches or leaves. While area (<b>d</b>) highlights small target recognition. Finally, area (<b>f</b>) presents cases of mutual occlusion between fruits.</p> Full article ">
