BIRDSAI: A Dataset for Detection and Tracking in Aerial Thermal Infrared Videos

Elizabeth Bondi, Raghav Jain, Palash Aggrawal, Saket Anand, Robert Hannaford, Ashish Kapoor, Jim Piavis, Shital Shah, Lucas Joppa, Bistra Dilkina, Milind TambeWACV 2020 · DOI

Abstract

Illegal poaching and animal trafficking represent a global crisis, particularly in remote protected areas where surveillance is difficult. While Unmanned Aerial Vehicles (UAVs) equipped with thermal infrared (TIR) sensors offer a solution, the manual analysis of this data is labor-intensive and prone to error. We present BIRDSAI (Benchmarking IR Dataset for Surveillance with Aerial Intelligence), a large-scale dataset designed to train and evaluate AI models for detecting and tracking humans and animals in aerial TIR footage. By combining real-world data from African conservation efforts with high-fidelity synthetic imagery, we demonstrate that synthetic-to-real domain adaptation can significantly improve detection performance in challenging nighttime environments.

Introduction

Conservationists increasingly rely on UAVs for nighttime surveillance to combat poaching. However, automating detection in aerial TIR videos faces significant hurdles: a lack of annotated thermal data, environmental noise (such as thermal reflections from the ground), and extreme scale variations due to drone altitude. BIRDSAI addresses these challenges by providing a diverse, annotated benchmark for the computer vision community.

The BIRDSAI Dataset

The core of this work is a hybrid dataset consisting of both real-world and synthetic components:

  • Real-World Data: 48 aerial TIR videos (~62,000 frames) collected by Air Shepherd across national parks in South Africa, Malawi, and Zimbabwe. These include annotations for giraffes, lions, elephants, and humans.
  • Synthetic Data: 124 videos (~100,000 frames) generated using Microsoft AirSim-W. This simulation environment models an African savanna, providing "perfect" ground-truth labels for training.
Comparison of real-world (a, b) and synthetic (c, d) TIR images in the BIRDSAI dataset.

As shown in the figure above, the synthetic data (c, d) closely mimics the thermal characteristics and environmental contexts of the real-world footage (a, b), providing a scalable way to augment training sets.

Methodology and Benchmarking

We evaluated three standard object detection architectures: Faster R-CNN, YOLOv2, and SSD. To address the significant class imbalance—where animals are far more frequent than humans—we utilized a Weighted Cross Entropy (WCE) loss function.

A key focus of our methodology was Unsupervised Domain Adaptation. This technique allows models trained on clean synthetic data to adapt to the noisy real-world domain without requiring additional manual labels for the real-world set.

Key Results and Insights

Detection Performance

Faster R-CNN with Weighted Cross Entropy achieved the highest overall performance with a mean Average Precision (mAP) of 0.438. Crucially, domain adaptation proved highly effective: leveraging synthetic data increased the mAP on real-world footage from 0.309 to 0.459.

The Challenge of Scale and Density

Analysis of the dataset reveals that models struggle most with small objects. While large animals like elephants are detected with relatively high accuracy, humans and smaller animals at high altitudes remain difficult to identify.

Statistical analysis of BIRDSAI: (a) Object size distribution, (b) Average object density, and (c) Sequence length vs. density.

As illustrated in the statistics above, the dataset contains a high percentage of small and medium objects, particularly in the human classes. Furthermore, the tracking performance (evaluated using the ECO tracker) dropped from 0.81 precision on ideal sequences to 0.48 on full, noisy sequences, highlighting the impact of sensor artifacts like motion blur and "ghosting."

Conclusion

BIRDSAI represents a significant step forward in providing the data necessary for AI-driven wildlife protection. Our results show that while synthetic data is a powerful "force multiplier" for training, significant work remains to improve the detection of small objects and the robustness of tracking algorithms. Future research should focus on edge-deployable architectures capable of real-time processing to provide immediate alerts to park rangers.

Summary updated

BibTeX
@inproceedings{bondi2020birdsai,
  author = {Elizabeth Bondi and Raghav Jain and Palash Aggrawal and Saket Anand and Robert Hannaford and Ashish Kapoor and Jim Piavis and Shital Shah and Lucas Joppa and Bistra Dilkina and Milind Tambe},
  title = {{BIRDSAI: A Dataset for Detection and Tracking in Aerial Thermal Infrared Videos}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2020},
  doi = {10.1109/wacv45572.2020.9093284},
}
Figure