Demo: Enhancing Wildlife Acoustic Data Annotation Efficiency through Transfer and Active Learning

Hannes Kath, Patricia P. Serafini, Ivan B. Campos, Thiago S. Gouvêa, Daniel Sonntag

Introduction

Passive Acoustic Monitoring (PAM) has become a cornerstone of modern conservation, allowing researchers to monitor biodiversity non-invasively by deploying autonomous recording units in remote habitats. However, the resulting "data deluge"—thousands of hours of audio recordings—presents a significant bottleneck. Manually identifying species within these recordings is a laborious, expert-intensive process that often delays critical conservation insights. While machine learning offers a path toward automation, high-performing models require vast amounts of labeled data, creating a "chicken and egg" problem for new or under-studied ecosystems.

Methodology: Breaking the Labeling Bottleneck

To address this challenge, we propose a semi-automatic workflow that integrates Transfer Learning and Active Learning to accelerate the annotation process.

Comparison of traditional manual annotation (left) vs. the proposed semi-automatic workflow (right).

Transfer Learning

Instead of training models from scratch, our approach leverages BirdNet, a deep learning architecture pre-trained on millions of bird vocalizations. By extracting "embeddings" (mathematical fingerprints) from 3-second audio snippets, the system gains an immediate understanding of acoustic features. This "head start" allows the tool to group similar sounds and identify patterns even for species not present in the original training set.

Active Learning

Rather than presenting samples in random order, the system employs Active Learning (AL) strategies. These algorithms prioritize samples where the model is most uncertain or those that represent the greatest acoustic diversity. By focusing human effort on these high-information samples, the underlying model improves more rapidly with fewer manual labels.

The YAPAT Annotation Tool

The proposed workflow is implemented in YAPAT, a no-code, web-based interface designed to augment, rather than replace, the domain expert.

The YAPAT web-based annotation interface showing snippet selection, spectrogram visualization, and model-assisted labeling.

The interface provides several features to reduce cognitive load:

  • Visual Representation: High-resolution spectrograms with band-pass filtering capabilities to isolate specific frequency ranges.
  • Semi-Automatic Labeling: The model suggests potential species labels, which the expert can confirm or correct with a single click.
  • Real-time Iteration: The model is updated iteratively as new labels are provided, creating a continuous feedback loop that improves suggestion accuracy during the session.

Evaluation and Results

The system was evaluated using a real-world dataset of oceanic bird vocalizations from Fernando de Noronha, a UNESCO World Natural Heritage site. Preliminary tests with domain experts demonstrated a 2–4x improvement in annotation speed compared to traditional manual methods.

In a recorded session, the median time to annotate a 3-second snippet was reduced to just 11 seconds. This efficiency significantly lowers the barrier to processing large-scale PAM datasets, bringing the ratio of annotation time to audio duration down to a manageable level for conservation projects.

Conclusion and Future Work

By combining the feature-extraction power of Transfer Learning with the strategic sampling of Active Learning, our tool transforms the "data deluge" into a structured learning process. Future developments will focus on integrating external sound libraries like Xeno-Canto for real-time reference and conducting broader user studies across diverse ecosystems, including tropical rainforests and marine environments.

Summary updated

BibTeX
@inproceedings{kath2024demo,
  author = {Hannes Kath and Patricia P. Serafini and Ivan B. Campos and Thiago S. Gouvêa and Daniel Sonntag},
  title = {{Demo: Enhancing Wildlife Acoustic Data Annotation Efficiency through Transfer and Active Learning}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2024},
}
Figure