Self-Supervised Pretraining and Controlled Augmentation Improve Rare Wildlife Recognition in UAV Images

Xiaochen Zheng, Benjamin Kellenberger, Rui Gong, Irena Hajnsek, Devis Tuia

Abstract

Automated wildlife monitoring using Unmanned Aerial Vehicles (UAVs) is hindered by the scarcity of labeled data for rare species and the vast, repetitive nature of aerial backgrounds. This paper presents a self-supervised learning (SSL) framework that leverages unlabeled domain-specific drone imagery to improve recognition performance. By integrating Momentum Contrast (MoCo) with Cross-Level Instance-Group Discrimination (CLD) and rotation-invariant augmentations, the proposed approach significantly reduces the reliance on manual annotations. Results on the Kuzikus Wildlife Dataset demonstrate that this method achieves superior accuracy compared to standard ImageNet-based models, particularly in low-data regimes where it doubles the recall of baseline models using only 10% of labeled data.

Introduction

Automated wildlife censuses using Unmanned Aerial Vehicles (UAVs) are vital for conservation, but they face a massive "data bottleneck." Wildlife is rare, and in high-resolution aerial shots, animals appear as tiny clusters of pixels against vast, repetitive backgrounds like savannas or forests.

Standard deep learning approaches typically rely on supervised pretraining on datasets like ImageNet. However, ImageNet consists of "object-centric" ground-level photos (e.g., a large dog in the center of the frame), which differ fundamentally from the "top-down" perspective of UAV imagery. Furthermore, manual annotation of thousands of tiny animals is tedious and expensive. This research seeks to reduce this labeling burden by leveraging the vast amounts of unlabeled UAV data through self-supervised learning (SSL).

The Challenge of Aerial Imagery

UAV imagery challenges: animals are small relative to the background, and background patches often share high similarity, leading to potential "false repulsion" in standard contrastive learning. Self-Supervised Pretraining and Controlled Augmentation Improve Rare Wildlife Recognition in UAV Images, page 1

UAV imagery challenges: animals are small relative to the background, and background patches often share high similarity, leading to potential "false repulsion" in standard contrastive learning.

As illustrated in Figure 1, the primary difficulty lies in the scale and similarity of features. Animals (foreground) are sparse, while background elements like grass and trees are abundant and visually similar across different regions. Standard contrastive learning often fails in this context because it attempts to distinguish between every unique patch, even if two patches of grass are virtually identical. This leads to "false repulsion," where the model wastes capacity trying to differentiate between semantically identical background elements.

Methodology: Adapting SSL for the Savanna

The researchers propose a framework that combines three key components to adapt SSL for the unique constraints of aerial imagery:

  1. Momentum Contrast (MoCo v2): A standard contrastive learning approach that trains a model to recognize that two different "views" (augmentations) of the same image patch should have similar representations, while views of different patches should be distinct.
  2. Cross-Level Instance-Group Discrimination (CLD): To address the false repulsion problem, CLD incorporates local K-Means clustering. This encourages the model to group similar-looking patches together (group-level) rather than forcing them apart at the instance level.
  3. Controlled Geometric Augmentation: Since UAVs look straight down, the "up" direction is arbitrary. The authors introduced random 90°, 180°, and 270° rotations during pretraining to bake this geometric understanding into the model, effectively providing a "free" signal for invariance.
The proposed SSL architecture combining MoCo (instance-level) and CLD (group-level) discrimination to learn robust features from unlabeled UAV imagery.

The final model, dubbed MCC2, was pretrained on unlabeled patches and then fine-tuned on a small set of labeled data for the final recognition task. As shown in Figure 2, the architecture utilizes dual projection heads to handle both instance-level similarity and group-level centroid alignment.

Datasets and Evaluation

The study utilizes the Kuzikus Wildlife Dataset, containing RGB aerial images from a Namibian wildlife reserve. Two subsets were derived:

  • KWD-Pre: A large, unlabeled dataset used for SSL pretraining, consisting of patches randomly cropped from original UAV images.
  • KWD-LT (Long-Tail): A labeled dataset used for evaluation. It reflects the real-world imbalance where background patches (trees, grass, roads) vastly outnumber foreground patches (animals).

Key Results

The authors compared their best SSL model (MCC2) against a standard supervised baseline (Sup2) pretrained on ImageNet:

  • Domain-Specific SSL Trumps Generic Supervision: Pretraining on "raw" unlabeled drone images proved more effective than pretraining on millions of labeled "natural" images. The model learns the specific textures and "look" of the savanna better than it learns from generic ground-level photos.
  • Massive Gains with Few Labels: The most striking result occurred when labels were scarce. With only 10% of the training labels, the MCC2 model achieved 84.5% recall, whereas the ImageNet baseline managed only 40.2% recall at similar precision.
  • Efficiency: The SSL approach effectively allows researchers to achieve the same performance as a standard model while performing 90% less manual labeling.

Conclusion and Future Work

This research demonstrates that self-supervised pretraining on domain-specific imagery is a powerful tool for overcoming the data bottleneck in wildlife conservation. By accounting for the high similarity of background patches and leveraging the geometric properties of top-down imagery, the MCC2 model provides a robust foundation for automated surveys. Future work could extend this framework from patch classification to full-image object detection, further streamlining the census process for endangered species.

Summary updated

BibTeX
@inproceedings{zheng2021self,
  author = {Xiaochen Zheng and Benjamin Kellenberger and Rui Gong and Irena Hajnsek and Devis Tuia},
  title = {{Self-Supervised Pretraining and Controlled Augmentation Improve Rare Wildlife Recognition in UAV Images}},
  booktitle = {ICCVW},
  year = {2021},
  doi = {10.1109/iccvw54120.2021.00087},
}
Figure