MinneApple: A Benchmark Dataset for Apple Detection and Segmentation

Nicolai Häni, Pravakar Roy, Volkan Isler

Abstract

In this work, we present a new dataset to advance the state-of-the-art in fruit detection, segmentation, and counting in orchard environments. While there has been significant recent interest in solving these problems, the lack of a unified dataset has made it difficult to compare results. We hope to enable direct comparisons by providing a large variety of high-resolution images acquired in orchards, together with human annotations of the fruit on trees. The fruits are labeled using polygonal masks for each object instance to aid in precise object detection, localization, and segmentation. Additionally, we provide data for patch-based counting of clustered fruits. Our dataset contains over 41,000 annotated object instances in 1,000 images. We present a detailed overview of the dataset together with baseline performance analysis for bounding box detection, segmentation, and fruit counting as well as representative results for yield estimation. We make this dataset publicly available and host a CodaLab challenge to encourage comparison of results on a common dataset.

Introduction

Automating fruit harvesting and yield mapping is a critical goal for precision agriculture, yet the orchard environment presents significant computer vision challenges. Unlike general-purpose datasets like COCO or ImageNet, orchard imagery is characterized by extreme occlusion from leaves and branches, highly variable lighting conditions, and significant scale issues where a single image may contain over 100 small fruit instances. Historically, research in this field has relied on small, private datasets, hindering the ability to objectively compare algorithmic performance. MinneApple addresses this by providing a large-scale, open-source benchmark.

The MinneApple Dataset

The MinneApple dataset is significantly denser than previous efforts in agricultural vision. It consists of 1,000 high-resolution images containing 41,325 annotated instances. For the counting task, the authors provide a separate dataset of 13,000 image patches for training and nearly 3,000 for testing, specifically curated to address the difficulty of identifying clustered fruit.

Examples from the MinneApple dataset illustrating the variety of fruit colors, densities, and orchard conditions, including segmentation and counting tasks.

While general datasets like COCO include an "apple" category, they typically average only 7.7 instances per image. In contrast, MinneApple averages 41.2 instances per image, with some scenes containing up to 120 apples, providing a much more rigorous test for object detection and segmentation models.

Methodology

Data was collected using a standard smartphone (Samsung Galaxy S4) at the University of Minnesota’s Horticultural Research Center to ensure the benchmark reflects real-world, low-cost sensor input. To support both object detection and instance segmentation, the authors moved beyond simple bounding boxes, providing precise polygonal masks for every fruit instance.

Detailed instance-level annotations used to support precise fruit detection and segmentation.

The study establishes baseline performance using several state-of-the-art architectures:

  • Detection: Faster R-CNN.
  • Segmentation: Mask R-CNN and UNet.
  • Counting: A ResNet-50 based Convolutional Neural Network (CNN) compared against a traditional Gaussian Mixture Model (GMM).

Key Insights and Results

  • The Small Object Challenge: The average apple size is approximately 40x40 pixels (0.17% of the image area). Standard detectors struggle with these small instances, marking this as a primary area for future architectural innovation.
  • Pretraining and Tiling: Models pretrained on ImageNet significantly outperformed those trained from scratch, suggesting that low-level features are highly transferable to agricultural contexts. Interestingly, image "tiling" (chopping images into smaller segments) actually degraded Faster R-CNN performance due to boundary artifacts.
  • Performance Floor: Faster R-CNN achieved an Average Precision (AP) of 0.438 for detection, while a pretrained UNet achieved an Intersection over Union (IoU) of 0.685 for segmentation. The CNN-based counting approach proved highly robust, reaching over 90% accuracy.
  • Yield Estimation: By combining detection and tracking, the system achieved between 95.5% and 97.8% accuracy compared to actual ground-truth harvest counts.

Conclusion and Future Work

MinneApple provides a standardized foundation for moving agricultural robotics toward robust, comparable solutions. Future work will need to address multi-object tracking (MOT) to prevent double-counting during camera movement, manage the extreme class imbalance between fruit and background pixels, and improve the detection of low-resolution or motion-blurred fruit.

To download the data and learn more, please visit the project website: http://rsn.cs.umn.edu/index.php/MinneApple.

Summary updated

Abstract

In this work, we present a new dataset to advance the state-of-the-art in fruit detection, segmentation, and counting in orchard environments. While there has been significant recent interest in solving these problems, the lack of a unified dataset has made it difficult to compare results. We hope to enable direct comparisons by providing a large variety of high-resolution images acquired in orchards, together with human annotations of the fruit on trees. The fruits are labeled using polygonal masks for each object instance to aid in precise object detection, localization, and segmentation. Additionally, we provide data for patch-based counting of clustered fruits. Our dataset contains over 41, 000 annotated object instances in 1000 images. We present a detailed overview of the dataset together with baseline performance analysis for bounding box detection, segmentation, and fruit counting as well as representative results for yield estimation. We make this dataset publicly available and host a CodaLab challenge to encourage comparison of results on a common dataset. To download the data and learn more about MinneApple please see the project website: http://rsn.cs.umn.edu/index.php/MinneApple. Up to date information is available online.
BibTeX
@article{hani2020minneapple,
  author = {Nicolai Häni and Pravakar Roy and Volkan Isler},
  title = {{MinneApple: A Benchmark Dataset for Apple Detection and Segmentation}},
  journal = {IEEE Robotics Autom. Lett.},
  year = {2020},
  url = {https://openreview.net/forum?id=2HA1ShhKFc},
}
Figure