Deep Learning Object Detection for Image Analysis of Cherry Fruit Fly (Rhagoletis cerasi L.) on Yellow Sticky Traps

Christian Salamut, Iris Kohnert, Niels Landwehr, Michael Pflanz, Michael Schirrmann, Mohammad ZareGesunde 2022 · DOI

TL;DR

Researchers have developed a deep learning-based pipeline to automate the tedious task of counting cherry fruit flies on yellow sticky traps. By evaluating several state-of-the-art object detection models, they achieved an Average Precision (AP) of 0.9, demonstrating that automated systems can match human accuracy while significantly increasing the scale and frequency of orchard monitoring.

What Problem Does This Paper Address?

In cherry orchards, the European cherry fruit fly (Rhagoletis cerasi) is a major pest that can devastate crops. To manage them, farmers use yellow sticky traps to monitor population levels. Currently, this monitoring is a manual "boots-on-the-ground" process: an expert must physically visit traps and count the flies.

This manual approach is laborious, time-consuming, and prone to human error. Because it costs so much in man-hours, farmers can only monitor a few locations, leaving large parts of the orchard unobserved. The authors aim to bridge this gap by using computer vision to turn high-resolution trap images into actionable data automatically.

Methods

The study compares five prominent deep learning architectures to find the best balance between accuracy and speed:

  1. Faster R-CNN (with ResNet and MobileNet backbones)
  2. Single Shot Detector (SSD)
  3. RetinaNet
  4. YOLOv5 (You Only Look Once)

While these are standard architectures in the computer vision world, the authors applied specific data augmentation strategies critical for agricultural contexts. Because insects are tiny relative to the size of a full sticky trap, the researchers used "grid cropping" and "sliding window cropping." This breaks a single high-resolution image into smaller patches, effectively "zooming in" so the neural networks can better learn the distinct features of the flies (like their wing patterns).

Datasets

The researchers built a custom image annotation database from traps in an orchard in eastern Germany:

  • 2020 Dataset: 140 high-resolution images featuring 47 different insect classes (multi-class).
  • 2021 Dataset: 850 images focused specifically on the cherry fruit fly (single-class).
  • Annotations: Over 1,600 individual cherry fruit flies were manually labeled.
  • Augmented Data: Through flipping and cropping, the training set was expanded significantly—reaching over 177,000 image patches for the sliding window method.

Key Insights and Ideas

  • The Resolution Paradox: High-resolution images are necessary to see small insects, but feeding a massive 3000x2000 pixel image directly into a model often leads to poor performance because the object of interest occupies only a tiny fraction of the total pixels. The "aha moment" here is that sliding window cropping isn't just a pre-processing step; it’s a fundamental requirement for detecting small biological features.
  • Architecture Sensitivity: Not all "state-of-the-art" models are created equal for this task. YOLOv5, often the darling of real-time detection, was outperformed by Faster R-CNN and RetinaNet in this study. This suggests that for static images where precision is more vital than raw frame rate, two-stage detectors (like Faster R-CNN) or models with specialized loss functions (like RetinaNet’s Focal Loss) handle the "small object" problem better.
  • Class Imbalance is the Real Enemy: In the multi-class 2020 dataset, performance dropped significantly (mAP of 0.51). This is because "indefinable insects" and background noise far outnumbered the specific pests, a common hurdle when moving from controlled lab settings to messy, real-world agricultural environments.

Key Results

  • Top Performers: Faster R-CNN (with MobileNet) and RetinaNet (with ResNet) emerged as the winners, both achieving an Average Precision (AP) of approximately 0.88 to 0.90 at a 0.5 Intersection over Union (IoU) threshold.
  • Inference Speed: The models processed images at 9–16 frames per second (fps). While this is slower than some industrial applications, it is more than fast enough for "near real-time" orchard monitoring.
  • Training Stability: Through a rigorous grid search of learning rates, the authors identified that a rate of $10^{-2}$ to $10^{-3}$ was the "sweet spot" for convergence; higher rates often led to "NaN" errors where the model failed to learn entirely.

Limitations and Future Work

The primary limitation is the computational cost of the sliding window approach. While it improves accuracy, processing every small patch of a large image individually slows down the system (dropping to ~0.34 fps in some configurations).

Future work will likely focus on:

  • Parallelization: Running the sliding window patches through the GPU simultaneously to regain speed.
  • Synthetic Data: Using Generative Adversarial Networks (GANs) to create more training examples for rare insect species to solve the class imbalance problem.
  • Edge Deployment: Optimizing these models to run on low-power devices that can be mounted directly in the field, providing real-time alerts to farmers' smartphones.

Summary updated

Abstract

Abstract Insect populations appear with a high spatial, temporal and type-specific diversity in orchards. One of the many monitoring tools for pest management is the manual assessment of sticky traps. However, this type of assessment is laborious and time-consuming so that only a few locations can be controlled in an orchard. The aim of this study is to test state-of-the art object detection algorithms from deep learning to automatically detect cherry fruit flies ( Rhagoletis cerasi ), a common insect pest in cherry plantations, within images from yellow sticky traps. An image annotation database was built with images taken from yellow sticky traps with more than 1600 annotated cherry fruit flies. For better handling in the computational algorithms, the images were augmented to smaller ones by the known image preparation methods “flipping” and “cropping” before performing the deep learning. Five deep learning image recognition models were tested including Faster Region-based Convolutional Neural Network (R-CNN) with two different methods of pretraining, Single Shot Detector (SSD), RetinaNet, and You Only Look Once version 5 (YOLOv5). R‑CNN and RetinaNet models outperformed other ones with a detection average precision of 0.9. The results indicate that deep learning can act as an integral component of an automated system for high-throughput assessment of pest insects in orchards. Therefore, this can reduce the time for repetitive and laborious trap assessment but also increase the observed amount of sticky traps
BibTeX
@inproceedings{salamut2022deep,
  author = {Christian Salamut and Iris Kohnert and Niels Landwehr and Michael Pflanz and Michael Schirrmann and Mohammad Zare},
  title = {{Deep Learning Object Detection for Image Analysis of Cherry Fruit Fly (Rhagoletis cerasi L.) on Yellow Sticky Traps}},
  booktitle = {Gesunde},
  year = {2022},
  doi = {10.1007/s10343-022-00794-0},
}
Figure