Research on an Apple Recognition and Yield Estimation Model Based on the Fusion of Improved YOLOv11 and DeepSORT

Zhibo Yan, Yu-Wei Wu, Wenbo Zhao, Shao Zhang, Xu LiAgriculture 2025 · DOI

TL;DR

This paper presents a novel apple detection and yield estimation system, APYOLO, which fuses an improved YOLOv11 with DeepSORT. By incorporating multi-scale channel attention (MSCA) and an enhanced loss function (EnMPDIoU), APYOLO significantly boosts detection accuracy in challenging orchard conditions, achieving an overall yield estimation accuracy of 84.45%.

What Problem Does This Paper Address?

Accurate apple yield estimation is crucial for efficient orchard management, market planning, and farmer income. However, traditional methods are labor-intensive, subjective, and not scalable. Modern approaches using computer vision and deep learning face significant challenges in complex orchard environments, particularly due to:

  • Dense Foliage Occlusion: Leaves and branches can hide apples, making them difficult to detect.
  • Overlapping Fruits: Apples growing close together can be mistaken for a single fruit or missed entirely.
  • Varying Lighting and Growth Stages: These factors further complicate accurate identification.

These issues lead to underestimation of yield, impacting the reliability of agricultural planning and economic forecasts.

Methods

The core of this research lies in enhancing existing object detection and tracking frameworks:

  • Improved YOLOv11 (APYOLO):
    • Multi-Scale Channel Attention (MSCA): This mechanism captures apple features across various scales and dimensions, helping the model adapt to different apple sizes and orchard conditions. It uses multi-scale pooling and a coordinate attention approach to better focus on relevant features.
  • Enhanced Multi-Scale Prior Distribution Intersection over Union (EnMPDIoU) Loss Function: This novel loss function improves bounding box regression by considering geometric terms like center distance and size differences. It aims to provide more accurate localization, especially for partially occluded or variably shaped apples, and addresses limitations of previous loss functions like isometric error penalization and convergence latency.
  • DeepSORT Tracking:
    • The APYOLO model is integrated with DeepSORT, a robust multi-object tracking framework.
  • Region of Line (ROL) Method: A line-crossing detection technique is incorporated into DeepSORT. This method counts targets as they cross a defined line, preventing double-counting and improving tracking accuracy, especially in scenarios with swaying branches or repeated crossings.

Datasets

  • Custom Apple Dataset: The study utilized a dataset comprising 3780 apple images and 10 video files collected from an orchard in Alar, Xinjiang, China. The data was captured using an iPhone 13 smartphone and includes variations in shooting angles, lighting conditions, and growth stages.
  • Data Pre-processing: The dataset underwent pre-processing including removal of similar images, grayscale transformation, normalization, rotation, brightness adjustment, and the addition of Gaussian noise and blurring to simulate diverse shooting conditions.
  • Data Split: The dataset was split into training (80%), validation (10%), and testing (10%) sets.

Key Insights and Ideas

  • Attention is Key for Occlusion: The MSCA mechanism's ability to extract features at multiple scales and focus on relevant channels is crucial for overcoming dense foliage and fruit overlap.
  • Geometric Loss for Better Localization: The EnMPDIoU loss function's focus on geometric relationships (center distance, size differences) offers a more precise way to define bounding boxes, especially for irregularly shaped or partially hidden apples.
  • Tracking for Reliable Counting: DeepSORT's robust tracking, combined with the ROL line method, provides a reliable mechanism for counting apples, preventing overcounting and improving overall yield estimation accuracy.
  • Trade-offs in Performance: While improvements in accuracy are significant, there's an inherent trade-off with increased model parameters and inference time, which needs careful consideration for real-time deployment.

Key Results

  • APYOLO vs. YOLOv11: APYOLO improved [email protected] by 2.2%, [email protected]–0.95 by 2.1%, accuracy by 0.8%, and recall by 2.3% compared to the original YOLOv11.
  • Overall Yield Estimation Accuracy: The combined APYOLO + DeepSORT system achieved a yield estimation accuracy of 84.45%.
  • Comparison with Other Models: APYOLO demonstrated competitive or superior performance against other YOLO variants (YOLOv5, YOLOv6, YOLOv8, YOLOv9t, YOLOv10n, YOLOv10s, YOLOv11) in terms of accuracy, recall, and mAP. It achieved the highest accuracy (85.0%) alongside YOLOv9t but surpassed it in recall.
  • Robustness: APYOLO showed strong performance in detecting apples under severe occlusion and effectively identified small targets in the distance.
  • Manual vs. Model Detection: A strong correlation (R² = 0.96556) was observed between manual apple counts and model-detected counts.
  • Error Rate: The model's error rate for yield estimation was as low as 1.4% in optimal conditions, well within the acceptable range for agricultural applications.

Limitations and Future Work

  • Computational Cost: The integration of MSCA and EnMPDIoU, while improving accuracy, leads to an increase in model parameters and inference time. This is a trade-off that needs careful management for real-time applications, especially on resource-constrained devices.
  • Sensitivity to Minor Movements/Occlusions: The ROL line mechanism, while improving accuracy, slightly reduces sensitivity to minor movements and occlusions, which could be a factor in very dense orchards.
  • Environmental Variations: The current dataset was collected under relatively stable conditions. Future work should focus on expanding the dataset to include diverse weather scenarios (fog, thunderstorms, sandstorms) and varying orchard layouts to improve robustness.
  • Multimodal Data Fusion: Integrating depth or infrared images could further enhance detection accuracy, particularly in low-light conditions or scenarios with significant occlusion.
  • Optimization for Edge Devices: Future research will explore lightweight algorithms, model pruning, distillation, and specific YOLO architecture optimizations for efficient deployment on edge and embedded devices.
  • Alternative Tracking Mechanisms: Exploring alternative tracking mechanisms could enhance the model's robustness and sensitivity in complex environments.

Summary updated

Abstract

Accurate apple yield estimation is essential for effective orchard management, market planning, and ensuring growers’ income. However, complex orchard conditions, such as dense foliage occlusion and overlapping fruits, present challenges to large-scale yield estimation. This study introduces APYOLO, an enhanced apple detection algorithm based on an improved YOLOv11, integrated with the DeepSORT tracking algorithm to improve both detection accuracy and operational speed. APYOLO incorporates a multi-scale channel attention (MSCA) mechanism and an enhanced multi-scale prior distribution intersection over union (EnMPDIoU) loss function to enhance target localization and recognition under complex environments. Experimental results demonstrate that APYOLO outperforms the original YOLOv11 by improving [email protected], [email protected]–0.95, accuracy, and recall by 2.2%, 2.1%, 0.8%, and 2.3%, respectively. Additionally, the combination of a unique ID with the region of line (ROL) strategy in DeepSORT further boosts yield estimation accuracy to 84.45%, surpassing the performance of the unique ID method alone. This study provides a more precise and efficient system for apple yield estimation, offering strong technical support for intelligent and refined orchard management.
BibTeX
@inproceedings{yan2025research,
  author = {Zhibo Yan and Yu-Wei Wu and Wenbo Zhao and Shao Zhang and Xu Li},
  title = {{Research on an Apple Recognition and Yield Estimation Model Based on the Fusion of Improved YOLOv11 and DeepSORT}},
  booktitle = {Agriculture},
  year = {2025},
  doi = {10.3390/agriculture15070765},
}
Figure