Pl@ntNet-300K image dataset

Camille Garcin, Alexis Joly, Pierre Bonnet, Antoine Affouard, Jean‐Christophe Lombardo, Mathias Chouet, Maximilien Servajean, Titouan Lorieul, Joseph Salmon2021 · DOI

Pl@ntNet-300K is a new large-scale dataset of 306,146 plant images across 1,081 species designed to move beyond "clean" benchmarks. By preserving the extreme class imbalance and high visual ambiguity found in the wild, it challenges researchers to develop "set-valued" classifiers that can handle real-world uncertainty where a single "correct" label is often impossible to determine.

The Challenge of Real-World Biodiversity Data

Most computer vision benchmarks, like ImageNet, are curated to be relatively balanced and distinct. However, real-world biodiversity data is "messy" in two specific ways:

  1. Aleatoric Uncertainty (Label Ambiguity): Many plant species look nearly identical, especially if the photo only captures a flower or a leaf. Even experts may require a list of candidate species rather than a single definitive answer.
  2. Epistemic Uncertainty (Long-tailed Distribution): In nature, a few species are everywhere, while most are rare. This leads to a "long tail" where models have very little data to learn from for the majority of classes.

The authors argue that standard "top-1" accuracy metrics fail to capture these nuances. They propose Pl@ntNet-300K as a testbed for set-valued classification—models that return a set of likely candidates—which is more useful for field applications like the Pl@ntNet mobile app.

Genus-Based Dataset Construction

The dataset was constructed using a "genus-based" sampling strategy. Rather than picking species at random, the authors sampled 10% of the genera from the massive Pl@ntNet citizen science database and kept all species within those genera. This ensures the dataset retains the high visual similarity (ambiguity) inherent to closely related plants.

Figure 4: The genus-based sampling strategy ensures that all species within a selected genus are included, preserving visual similarity and classification difficulty.

Evaluation Metrics for Uncertainty

To evaluate models on this difficult data, the authors focus on:

  • Set-valued Classification: Moving away from predicting one label to predicting a set $\Gamma(x)$.
  • Macro-averaging: Instead of calculating accuracy across all images (which favors common species), they calculate accuracy per species and then average those results. This forces the model to perform well on rare species to achieve a high score.
  • Average-k Accuracy: A metric that allows the size of the predicted set to vary depending on the model's confidence, provided the average set size across the dataset remains below a threshold $k$.

Key Insights and Results

The authors provided baselines using a wide array of architectures, from ResNets and DenseNets to Vision Transformers (ViT).

  • The "Expert Gap": The authors asked a professional botanist to label a subset of the data. The botanist had a 20.5% error rate while returning an average of 4.1 species per image. This confirms that for many plant images, "perfect" top-1 accuracy is an unrealistic goal due to the lack of discriminative information in the photo.
  • Performance Ceiling: While these models can reach ~80% macro-average top-1 on ImageNet, none exceeded 40% on Pl@ntNet-300K.
  • Model Comparison: Interestingly, models that perform similarly on ImageNet (like Inception-v3 and DenseNet-201) showed significantly different performance on this dataset, suggesting that Pl@ntNet-300K reveals different model strengths/weaknesses regarding ambiguity and imbalance.
  • Calibration: By calibrating model thresholds to match the botanist's average of 4.1 species returned, the best models still lagged behind the human expert's error rate, showing significant room for algorithmic improvement.

Limitations and Future Work

Currently, the paper treats all 1,081 species as independent classes. Future work could exploit the taxonomic hierarchy (genus, family) to improve results. Additionally, the authors note that there are currently no loss functions specifically designed to optimize "average-k" accuracy. They hope this dataset encourages the development of new training objectives that specifically target set-valued performance on long-tailed data.

Summary updated

BibTeX
@misc{garcin2021pl,
  author = {Camille Garcin and Alexis Joly and Pierre Bonnet and Antoine Affouard and Jean‐Christophe Lombardo and Mathias Chouet and Maximilien Servajean and Titouan Lorieul and Joseph Salmon},
  title = {{Pl@ntNet-300K image dataset}},
  year = {2021},
  doi = {10.5281/zenodo.5645731},
}

Loading related content...

Figure