Event-Triggered Visual Perception for Real-Time Harvest Outcome Estimation in Mechanical Blueberry Harvesting

Nicolas Miroslav Husár, Jelle Driessen, Petra Bosilj, Başak SakçakarXiv 2026 · arXiv

TL;DR

Researchers have developed an event-triggered computer vision system that provides real-time maturity feedback for mechanical blueberry harvesters. By running a three-class YOLO detector on a standard industrial CPU, the system accurately estimates the ripe-fruit fraction of harvested crates in under five seconds, enabling future closed-loop control of harvest selectivity.

What Problem Does This Paper Address?

Mechanical blueberry harvesting is a delicate balancing act. If the machine shakes the bushes too vigorously, it collects unripe berries that should have stayed for a later harvest; too gently, and ripe fruit is left behind. Currently, operators fly blind—they don't know the "maturity composition" of their haul until it reaches a commercial sorting facility hours or days later.

The challenge is bringing high-accuracy visual perception into the field. Agricultural environments are harsh, with vibrating machinery and unpredictable lighting. Furthermore, most modern computer vision models require power-hungry GPUs, which are rarely found on the rugged, CPU-only industrial computers typically used in farm equipment. This paper tackles the need for a "low-power, high-accuracy" solution that works in real-time during the harvest.

Methods

The authors implemented an event-triggered perception pipeline on a FineField Harvy 500 harvester. Instead of processing a continuous video stream—which would overwhelm the onboard Intel Celeron J1900 CPU—the system only captures and analyzes an image when a sensor detects a crate has been filled and moved.

Figure 1: FineField Harvy 500 mechanical blueberry harvester operating in a blueberry field. Photograph taken on a different day from the reported field experiment.

Key technical components include: * Detection Model: They used YOLO26s, a lightweight object detector. The training was split into two phases: first, a single-class detector to locate blueberries, followed by a three-class refinement to categorize them as ripe, semi-ripe, or unripe. * Optimization: The model was exported to OpenVINO to maximize inference speed on the harvester's quad-core CPU. * Hardware Integration: Two 5 MP global-shutter cameras were mounted 20 cm above the crate stream. Global shutters are crucial here to prevent image distortion caused by the harvester's constant vibration.

Datasets

The model was trained and evaluated using a mix of public and proprietary data:

  • Training: The Blueberry Detection Dataset (OPENset) and the Blueberry Maturity Classification dataset provided the foundation. These were supplemented with custom images from FineField to account for the dense, overlapping clusters seen in harvest crates.
  • Evaluation: The system was field-tested using 79 crate images containing over 48,000 manually annotated blueberries. This "ground truth" allowed for a rigorous comparison between the AI's estimates and human verification.

Key Insights and Ideas

  • Maturity Ratios vs. Absolute Counts: One of the most significant findings was that while absolute counting accuracy dropped under direct sunlight (due to overexposure), the ratio of ripe to unripe fruit remained remarkably stable. For harvest selectivity, the ratio is the more important metric.
  • The 14-Second Window: The harvester produces a crate roughly every 14 seconds per side. By using event-triggering, the system achieved a median trigger-to-result latency of 5.84 seconds. This proves that expensive GPUs aren't strictly necessary for sophisticated agricultural AI if the pipeline is architected efficiently.
  • Harvest Selectivity: The authors frame maturity composition as a component of "harvest selectivity." This shifts the focus from simple yield (how much did we get?) to quality (did we get the right berries?).
Figure 2: Field context of the study: (a) blueberry crop entering the mechanical picking zone; (b) machine-mounted camera positioned above the crate stream; and (c) harvested product showing ripe, semi-ripe, unripe fruit and debris.

Key Results

The system's performance was highly competitive with manual inspection:

  • Accuracy: The vision system estimated an overall ripe-fruit fraction of 90.22%, nearly identical to the manually determined 90.97%.
  • Precision: At the individual crate level, the Mean Absolute Error (MAE) was just 0.935 percentage points.
  • Correlation: A Pearson correlation of r = 0.967 indicates a very strong linear relationship between the AI's predictions and the ground truth.
  • Throughput: All 79 field events were processed within the required time window, demonstrating the system's readiness for real-world deployment.
A correlation plot showing the tight agreement between predicted and manual ripe-fruit fractions.

Limitations and Future Work

  • The "Top-Layer" Problem: The cameras only see the top layer of berries in a crate (roughly 23% of the total volume). While the authors argue this is a representative sample, future work is needed to confirm if the maturity composition is consistent throughout the depth of the crate.
  • Lighting Sensitivity: Although the maturity ratio was stable, the absolute count errors in direct sunlight suggest that adding physical shrouds or dedicated LED lighting would further improve robustness.
  • Closing the Loop: This study focused on estimation. The next logical step is automation: using this real-time data to automatically adjust the harvester's shaker frequency or ground speed to optimize the harvest outcome on the fly.
Figure 3: Example of the system’s saved detection output for a white-side crate. Green, yellow, and red bounding boxes indicate ripe, semi-ripe, and unripe blueberries, respectively. The displayed “PACKOUT” value of 84.8% is the detected ripe-fruit fraction (525 of 619 detections), not commercial Class I packout.

Summary updated

Abstract

Mechanical blueberry harvesting requires a balance between removing ripe fruit and avoiding premature removal of unripe berries that could remain on the bush for a later harvest. However, operators generally lack real-time information about the maturity composition of the fruit being collected. This study develops an event-triggered machine-vision system that measures the maturity of harvested blueberries directly on the harvester. When a filled crate is moved by the machine, an RGB image is automatically captured and analysed using a three-class detector that identifies ripe, semi-ripe, and unripe berries. The system operates locally on the existing CPU-only industrial computer and therefore requires neither continuous image processing nor dedicated GPU hardware. During field testing, 79 crate images containing 48,634 visible blueberries in manually verified reference annotations were evaluated. The vision system estimated an overall ripe-fruit fraction of 90.22%, compared with 90.97% determined manually. Per-image ripe-fruit estimates showed a mean absolute error of 0.935 percentage points and a Pearson correlation of r=0.967. Maturity estimates remained similar between the two sides of the harvester for which the light conditions varied despite poorer absolute counting performance under direct sunlight. All field events were processed within the available 14 s per-side crate interval. The proposed approach provides a practical means of quantifying one component of harvest selectivity while the machine is operating. Such information could support operator decision-making and, when combined with measurements of fruit recovery and losses, contribute to future feedback-based control of mechanical harvesting.
BibTeX
@misc{husar2026event,
  author = {Nicolas Miroslav Husár and Jelle Driessen and Petra Bosilj and Başak Sakçak},
  title = {{Event-Triggered Visual Perception for Real-Time Harvest Outcome Estimation in Mechanical Blueberry Harvesting}},
  year = {2026},
  eprint = {2611.21472},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.21472},
}
Figure