Overview of BirdCLEF 2022: Endangered bird species recognition in soundscape recordings

Stefan Kahl, Amanda Navine, Tom Denton, Holger Klinck, Patrick Hart, Hervé Glotin, Hervé Goëau, Willem-Pier Vellinga, Robert Planqué, Alexis Joly

Introduction

Hawai‘i is often referred to as the “extinction capital of the world,” having lost approximately 68% of its native bird species. For the remaining populations, consistent monitoring is vital for conservation efforts. However, traditional physical surveys in remote and dense tropical forests are logistically challenging and prohibitively expensive. Passive Acoustic Monitoring (PAM)—the deployment of autonomous recording units in the wild—offers a cost-effective alternative but generates a massive data bottleneck, requiring thousands of hours of manual expert review.

Target Hawaiian bird species for the BirdCLEF 2022 challenge, including (a) Nēnē, (b) ‘Akiapōlā‘au, (c) ‘Alawī, and (d) ‘I‘iwi.

The BirdCLEF 2022 challenge was designed to address this bottleneck by automating the identification of rare and endangered Hawaiian bird species within noisy, real-world soundscapes. The competition specifically focused on the "long-tail" problem in bioacoustics: identifying species with very limited training data amidst complex environmental backgrounds.

Methodology

The competition, hosted on Kaggle, attracted over 1,000 participants tasked with a multi-label classification problem: identifying which of 21 target species were vocalizing in 5-second segments of continuous audio.

Top-performing teams primarily utilized ensembles of Convolutional Neural Networks (CNNs). The EfficientNet-B0 architecture emerged as a preferred backbone due to its optimal balance of computational efficiency and classification performance. To bridge the gap between clean training clips and noisy field recordings, participants employed intensive data augmentation strategies, including Gaussian noise injection, loudness normalization, "mix-up" (overlaying different audio clips), and "spec augment."

Beyond standard architectures, some teams explored Audio Transformers and LSTMs to capture sequential patterns, though results were mixed. Sophisticated post-processing, such as quantile-based class-wise thresholding, was used to fine-tune model sensitivity for individual species, which is critical when dealing with imbalanced datasets.

Datasets

The challenge leveraged two primary data sources:

  1. Training Data: Approximately 14,800 "focal" recordings from the Xeno-canto community. These are high-quality, clean clips of specific birds covering 152 species.
  2. Test Data: A benchmark set of 5,356 one-minute soundscapes (~90 hours) provided by the Listening Observatory for Hawaiian Ecosystems (LOHE). These recordings represent real-world acoustic environments across seven sites on Hawai‘i, Maui, and Kaua‘i, featuring significant background noise and varying weather conditions.

Results and Discussion

The competition saw significant participation, with 801 teams from 62 countries submitting over 23,000 runs. While the baseline score (predicting silence for all segments) was a weighted F1 of 0.5112, the winning submission achieved a score of 0.8527. The use of a weighted macro-averaged F1-score ensured that models were penalized more for missing rare species, prioritizing recall for conservation needs.

Key Insights

  • Complexity over Quantity: Model performance showed only a weak correlation with the volume of training data. Success was driven more by the distinctiveness and repertoire size of a species' song than by the number of samples available.
  • The "Co-firing" Problem: Models frequently struggled with species that have similar vocalizations. For instance, the ‘I‘iwi and ‘Apapane (two native honeycreepers) are difficult even for experts to distinguish. Top models "co-fired"—predicting both species simultaneously—68% of the time when either was present.
The ‘I‘iwi (Drepanis coccinea), a native honeycreeper whose vocalizations are often confused with the ‘Apapane by automated models.
  • The False Positive Trap: Models exhibited a tendency to "over-fire," predicting bird activity in silent segments. While only one-third of the test segments contained target species, top models predicted at least one label for 94% of segments. This indicates a critical need for robust "voice activity detection" (VAD) to filter out silence before species classification.

Conclusion and Future Work

While BirdCLEF 2022 demonstrated the power of deep learning for bioacoustics, the models are not yet fully autonomous for conservation. The high rate of false positives and co-firing means that expert verification is still necessary. Future research should focus on few-shot learning for species with minimal recordings, improved upstream filtering to reduce false positives, and better domain adaptation to help models generalize from clean focal recordings to the complex acoustic environments of the Hawaiian rainforest.

Summary updated

Abstract

As the “extinction capital of the world”, Hawai'i has lost 68% of its native bird species, the consequences of which can harm entire ecosystems. With physical monitoring difficult, scientists have turned to sound recordings, as this approach could provide a passive, low labor, and cost-effective strategy for monitoring endangered bird populations. Current methods for processing large bioacoustic datasets involve manual review of each recording. This requires specialized training and prohibitively large amounts of time. Recent advances in machine learning have made it possible to automatically identify bird songs for common species with ample training data. However, it remains challenging to develop such tools for rare and endangered species. The main goal of the 2022 edition of BirdCLEF was to advance automated detection of rare and endangered bird species that lack large amounts of training data. The competition challenged participants to develop reliable analysis frameworks to detect and identify the vocalizations of rare bird species in continuous Hawaiian soundscapes utilizing limited training data.
BibTeX
@inproceedings{kahl2022overview,
  author = {Stefan Kahl and Amanda Navine and Tom Denton and Holger Klinck and Patrick Hart and Hervé Glotin and Hervé Goëau and Willem-Pier Vellinga and Robert Planqué and Alexis Joly},
  title = {{Overview of BirdCLEF 2022: Endangered bird species recognition in soundscape recordings}},
  booktitle = {CLEF (Working Notes)},
  year = {2022},
  url = {https://openreview.net/forum?id=uDRVu8kV0j},
}
Figure