Foundation models for bioacoustics - A comparative review

Raphael Schwinger, Paria Vali Zadeh, Lukas Rauch, Mats Kurz, Tom Hauschild, Sam Lapp, Sven Tomforde

This review paper provides a timely and rigorous look at the rapidly evolving landscape of "foundation models" for bioacoustics. As researchers increasingly turn to Passive Acoustic Monitoring (PAM) to track biodiversity, the bottleneck has shifted from data collection to data annotation. This paper systematically evaluates how large-scale pretrained models can bridge that gap.

TL;DR

This paper presents the first comprehensive comparative review of bioacoustic foundation models, evaluating 12+ architectures across the BEANS and BirdSet benchmarks. The authors demonstrate that self-supervised models like BirdMAE and BEATsNLM currently set the state-of-the-art, but only when paired with "attentive probing" to unlock their complex learned representations.

What Problem Does This Paper Address?

Monitoring biodiversity is critical for conservation, but manual analysis of thousands of hours of field recordings is impossible. While Deep Learning (DL) has revolutionized species identification, standard models struggle with "the long tail"—rare or under-studied species with very little training data.

Foundation models—models trained on massive, diverse datasets (like millions of bird recordings from Xeno-Canto)—offer a solution through transfer learning. However, the field lacks a "Rosetta Stone" to compare these models. Different models use different architectures (CNNs vs. Transformers), training paradigms (Supervised vs. Self-Supervised), and preprocessing steps. This paper provides a standardized benchmark to see which "backbone" actually generalizes best to new, unseen bioacoustic tasks.

Methods

The authors categorize and evaluate models based on several design dimensions:

  • Architectures: Comparing traditional Convolutional Neural Networks (CNNs) like ConvNext and EfficientNet against modern Transformer-based Vision Transformers (ViT).
  • Training Paradigms: Comparing Supervised Learning (SL) (training on labeled species) vs. Self-Supervised Learning (SSL) (pretext tasks like Masked Autoencoding or Masked Language Modeling).
  • Probing Strategies: This is a key technical contribution. They compare Linear Probing (training a single layer on top of a frozen model) vs. Attentive Probing (using a trainable attention mechanism to query the frozen model’s internal features).
  • Standardization: They re-evaluated models using consistent preprocessing (resampling, segment lengths, and spectrogram parameters) to ensure a fair "apples-to-apples" comparison.

Datasets

The study leverages a massive array of data:

  • Pretraining Sources: General audio (AudioSet), bird-specific repositories (Xeno-Canto, Macaulay Library), and citizen science data (iNaturalist).
  • Evaluation Benchmarks:
    • BEANS: A diverse benchmark covering multiple taxa (birds, bats, dogs, frogs, insects) and tasks.
  • BirdSet: A large-scale benchmark focused on avian species classification in both focal recordings and noisy "soundscape" (PAM) data.

Key Insights and Ideas

  • SSL is the New Gold Standard: Models trained with self-supervision (like BirdMAE) generally outperform supervised models when transferring to new tasks. SSL allows the model to learn the "physics" of sound without being constrained by specific taxonomic labels.
  • The "Attention Tax": Transformer-based models are powerful but "lazy" when it comes to linear probing. To extract their full potential, you must use attentive probing. Linear probing alone often fails to capture the complex temporal-spectral relationships Transformers learn.
  • Generalists vs. Specialists: Interestingly, general-purpose audio models (like BEATs, trained on YouTube clips) outperform bird-specific models on the general BEANS benchmark. However, for specialized bird tasks, domain-specific pretraining (like BirdMAE) still reigns supreme.
  • Linear Probing > Direct Evaluation: Even if a model was already trained on bird species, training a new linear classifier on its embeddings for a specific local task yields significantly better results than using the model's original "out-of-the-box" predictions.

Key Results

  • BirdMAE emerged as the top performer on the BirdSet benchmark (86.54 AUROC), showcasing the power of Masked Autoencoders for bioacoustics.
  • BEATsNLM (the encoder from NatureLM) took the crown on BEANS (98.57 AUROC), proving that "scaling up" with massive mixed-source data pays off for general animal sound recognition.
  • Efficiency: While Transformers won on raw performance, CNN-based models like Perch and ConvNextBS remained highly competitive and much more parameter-efficient for simple linear probing.

Limitations and Future Work

The authors acknowledge that they evaluated models only as frozen feature extractors. While this is standard for assessing representation quality, "fine-tuning" the entire model might yield even higher performance, albeit at a much higher computational cost.

Future research directions suggested include:

  1. Hybrid Pretraining: Combining general audio and bioacoustic data in a single pretraining stage.
  2. Advanced Adaptation: Exploring parameter-efficient fine-tuning (PEFT) methods like LoRA (Low-Rank Adaptation) specifically for bioacoustic tasks.
  3. PAM-Specific Models: Most current models are trained on "clean" focal recordings; there is a massive opportunity to build foundation models trained directly on raw, noisy soundscape data.

Summary updated

Abstract

Automated bioacoustic analysis is essential for biodiversity monitoring and conservation, requiring advanced deep learning models that can adapt to diverse bioacoustic tasks. This article presents a comprehensive review of large-scale pretrained bioacoustic foundation models and systematically investigates their transferability across multiple bioacoustic classification tasks. We overview bioacoustic representation learning including major pretraining data sources and benchmarks. On this basis, we review bioacoustic foundation models by thoroughly analysing design decisions such as model architecture, pretraining scheme, and training paradigm. Additionally, we evaluate selected foundation models on classification tasks from the BEANS and BirdSet benchmarks, comparing the generalisability of learned representations under both linear and attentive probing strategies. Our comprehensive experimental analysis reveals that BirdMAE, trained on large-scale bird song data with a self-supervised objective, achieves the best performance on the BirdSet benchmark. On BEANS, BEATs$_{NLM}$, the extracted encoder of the NatureLM-audio large audio model, is slightly better. Both transformer-based models require attentive probing to extract the full performance of their representations. ConvNext$_{BS}$ and Perch models trained with supervision on large-scale bird song data remain competitive for passive acoustic monitoring classification tasks of BirdSet in linear probing settings. Training a new linear classifier has clear advantages over evaluating these models without further training. While on BEANS, the baseline model BEATs trained with self-supervision on AudioSet outperforms bird-specific models when evaluated with attentive probing. These findings provide valuable guidance for practitioners selecting appropriate models to adapt them to new bioacoustic classification tasks via probing.
BibTeX
@article{schwinger2026foundation,
  author = {Raphael Schwinger and Paria Vali Zadeh and Lukas Rauch and Mats Kurz and Tom Hauschild and Sam Lapp and Sven Tomforde},
  title = {{Foundation models for bioacoustics - A comparative review}},
  journal = {Ecol. Informatics},
  year = {2026},
  doi = {10.1016/j.ecoinf.2026.103765},
}

Loading related content...

Figure