WildlifeDatasets: An Open-Source Toolkit for Animal Re-Identification

Vojtěch Čermák, Lukas Picek, Lukáš Adam, Kostas PapafitsorosWACV 2024

TL;DR

This paper introduces WildlifeDatasets, a comprehensive Python toolkit that simplifies access to and analysis of wildlife re-identification datasets. It also presents MegaDescriptor, a novel foundation model that achieves state-of-the-art performance across various species, outperforming existing general-purpose models.

What Problem Does This Paper Address?

Accurately identifying individual animals in the wild is crucial for ecological research, enabling population monitoring, behavioral studies, and conservation efforts. However, the field of animal re-identification faces significant challenges: a lack of standardized methods and datasets, poor dataset curation leading to data leakage, and a tendency for research to focus on single species, hindering generalizability and reproducibility. This makes it difficult for researchers to compare approaches and apply them effectively to real-world scenarios.

Methods

The core contribution is the WildlifeDatasets toolkit, a Python library designed for both ecologists and computer vision researchers. It offers:

  • Unified Dataset Access: A single interface to download, extract, and pre-process 31 publicly available wildlife re-identification datasets.
  • Advanced Dataset Splitting: Implementations for various splitting strategies (closed-set, open-set, disjoint-set) and a novel "time-aware" split to better reflect real-world ecological scenarios.
  • Feature Extraction and Matching: Support for local descriptors (SIFT, Superpoint), pre-trained deep learning models (CNNs and transformers via the Timm library), and the newly proposed MegaDescriptor model.
  • MegaDescriptor Model: This is a novel foundation model for individual re-identification. It's based on Swin transformer architectures and trained using metric learning techniques (specifically, ArcFace loss). The authors explored various backbone architectures (Swin-B vs. EfficientNet-B3) and metric learning losses (ArcFace vs. Triplet loss), finding that Swin-B with ArcFace generally performed best. They also conducted extensive hyperparameter tuning to find optimal settings.

Datasets

The toolkit provides access to 31 publicly available wildlife re-identification datasets. For training and evaluation of the MegaDescriptor model and ablation studies, the authors utilized a curated set of 29 datasets. They specifically mention using cattle datasets (AerialCattle2017, FriesianCattle2015, FriesianCattle2017, Cows2021, and OpenCows2020) to evaluate domain generalization capabilities.

Key Insights and Ideas

  • The Power of Foundation Models for Wildlife: The success of MegaDescriptor demonstrates that general-purpose foundation models, when properly adapted and trained on diverse wildlife data, can achieve superior performance compared to models trained on general image datasets (like ImageNet) or even other large vision-language models (like CLIP and DINOv2).
  • Metric Learning Outperforms Local Features: For most datasets, metric learning approaches (ArcFace and Triplet loss) significantly outperformed traditional local feature-based methods (SIFT and Superpoint).
  • Swin Transformers Shine: The Swin transformer backbone generally yielded better results than the EfficientNet-B3 CNN backbone when used with metric learning.
  • Domain Generalization is Key: MegaDescriptor shows promising generalization capabilities, performing well even on datasets it wasn't explicitly trained on, suggesting its learned features are robust and transferable across species.
  • Time-Aware Splits Matter: The introduction of time-aware dataset splits is a crucial step towards more ecologically realistic evaluations, accounting for temporal changes in populations.

Key Results

  • MegaDescriptor's Superior Performance: The MegaDescriptor-B (Swin-B backbone) achieved state-of-the-art performance on all 29 tested datasets, consistently outperforming CLIP, DINOv2, and ImageNet-pretrained models. For instance, on the AAAZebraFish dataset, MegaDescriptor achieved 99.93% accuracy, significantly higher than DINOv2 (96.93%) and CLIP (94.91%).
  • Metric Learning Dominance: In metric learning vs. local features comparisons, metric learning methods achieved accuracies often exceeding 90%, while local features struggled to reach 70% on many datasets.
  • ArcFace Consistency: ArcFace loss generally provided more consistent performance across datasets compared to Triplet loss.
  • Hyperparameter Sensitivity: The performance of metric learning approaches was found to be sensitive to hyperparameter choices, necessitating thorough grid search.

Limitations and Future Work

  • Dataset Curation: While the toolkit aims to unify datasets, the original datasets may still contain issues like data leakage, which the authors acknowledge as a persistent problem in the field.
  • Computational Resources: Training and evaluating models on such a large number of datasets and hyperparameter combinations is computationally intensive.
  • Future Work: The authors encourage community contributions to expand the toolkit with new datasets and methods. They also envision further development and expansion of the MegaDescriptor model. The toolkit's design emphasizes extensibility, allowing for the integration of new algorithms and datasets.

Summary updated

BibTeX
@inproceedings{cermak2024wildlifedatasets,
  author = {Vojtěch Čermák and Lukas Picek and Lukáš Adam and Kostas Papafitsoros},
  title = {{WildlifeDatasets: An Open-Source Toolkit for Animal Re-Identification}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2024},
}
Figure